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ABSTRACT 

This is the last of five documents compiled to report 
on the problem of innovation and change in the context of projects 
supported by the National Center for the Improvement of Educational 
Systems (NCIES) and the second of two documents tl:dt examine in 
detail the Career Opportunities Program (COP) and its effects on COP 
aides, students, and the institutions involved. Included in this 
document are discussions on survey design, the development of 
achievement tests, and the development of a student opinion 
questionnaire. Copies of all surveys and tests are included. A 
related document is EA 004 861. (Page 142 may reproduce poorly.) 
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INTRODUCTION 



This volume documents the research design, instruments, 
and field procedures employed in this study. 

The operational plan,, submitted on July 29, 1971 was ap- 
proved by the Office of Education and functioned as the plan 
of the study. The original research design which this document 
presented was slightly modified but the thrust of the study re- 
mained the same. 

The design was essentially a two-wave survey of COP aides 
and a multi-faceted survey of selected respondent groups who po- 
tentially could be impacted by the program. In order to assess 
impacts on diverse respondent groups, nine distinct questionnaires/ 
instruments were developed by Abt Assoc ates Inc. m addition, 
three standardized achievement tests -ere selected ' for croup 
testing of COP aides and students, gr 'es 1-6. One inst- .ment, 
the Student Opinionnaire , requ^ .red e.x . ::n- -^e c evel oprr, . n ^ . assure 
a satisfactory measurement procedure. .hese developm alorg 
with the instruments and justification statements submitted to 
0MB, are included in this volume. 

The field procedures in a survey are critical to the suc- 
cess of the study. In the case of this evaluation, great atten- 
tion was paid to this part of the study not only because strong 
field control is methodologically essential but also because we 
were concerned lest the project directors of the sites be un- 
sympathetic and therefore uncooperative. Some of the procedures 
required are not popular in many educational circles, i.e., 
achievement tests; some of the questions we would be asking -vere " 
somewhat sensitive ones, i.e., attitudes. Finally, since we had 
to return to the sites, there was always the danger that we would 
wear out our welcome. m consultation with the Office of Edu- 
cation we embarked on a planned program to fully explain the 
study and to actively seek the cooperation of the Project Directors. 
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Examples of our efforts alone this line were : 

• Special Washington, D.C- conference with the 16 Project 
Directors arranged and pciid for bi' the contractor. 

m Personal visits to those sites which included student 

testing to discuss the needs and instruments developed. 
The Research Director vifsited four of these sites 
between the fall and spring waves for the express 
purpose of explaining the uses to which the data would 
be put and modifying the procedures so far as pos ^i. • 
to satisfy t :e concerns of the sites. For example , 
one site objected to printing the fac3S of the Studer 
Opinionnaire j_n£trument on white paper since this does 
not take into account minority children , At their 
suggestj.on the color was changec to green. 

The Project Directors of all the s^-es were both cordial 
en- cooperative i r both the fall and spr „ .g field efforts. It 
must be recognized, however, that the COP program operates within 
the local constraints of the communities it serves. In several 
cases, the requirements for student testing put severe strains 
on the local COP staff ranging from the challenge of scheduling 
testing in the classrooms to local rules which prevented the dis- 
semination of information about students that was needed for the 
analysis . 

Although many programs are federally funded, they still 
must operate within specific regions, states, and localities in 
this country. These jurisdictions are not subservient to the 
federal government indeed, states rights are an integral part of 
the constitution. Recognizing this and from our own experiences 
in the fie]'^ we urge the Office of Education to reconsider its 
procedures in making evaluation decisions for its programs. That 
is, unless local conditions are considered, the data requests made 
at the federal level may not be able to be honored at the local 
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level. Responding to demands that cannot be easily met ^ if 
at all, places very heavy responsibility on the evaluation con- 
tractor, a responsibility that may not be appropriate to his 
function. We are suggesting that in the future v/h^ve data re-- 
quested by the Office of Education is perceived by the local site: 
as being "sensitive," the Office of Educatior use its persuasive 
powers to obtain such data or a leasT smooth -he way. 
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I. OPERATIONAL PLAN 



Introduction 

The essence of Abt Associates' plan is a dichotomized study 
of the impact of the eight BEPD programs identified as subjects 
for inve-- tigation i.i both the FP and - nr propose The first 
part is in the form of an anal cical survey of th impact of the 
Career Opportunities Program i the areas of institutional change, 
partic _pant attituu-.- and behav. , and student a ch_L-3 cement and 
behavior. The otheir part will be a series of c:.3e rtudies of the 
acTivi iies and imperrts of exerr.^lary proj:-cts an pre 7 '-:t sites 
c ver. from a].l eignt subject lE^D progr:.::us. ' iie f : -:f tr.-r 

s ddies will be on the ^ ::ri,iional nairc or tn^ ^;ar:-Cas 
H " srrati-":ies used and the resulting impacts of those strate- 
gies. The following sections of this plan discuss the two parts 
of the study in detail, as well as items of overall concern. 

The plan delineated here arose out of the obligatory re- 
solution of the ambiguity inherent in the specification of sam- 
pling procedures as they appear in different places within the 
contract between the Office of Education and Abt Associates Inc. 
The nature of the resolution addresses the expressed future 
direc^Lion of the Bureau of Educational Personnel Development. The 
present operational structure of the Bureau that provides for a 
number of discrete programs to upgrade educational personnel and 
the systems within which they work will be changing in the near 
future. The expectation is that only the Career Opportunities 
Program and the Teacher Corps Program ^vill retain their present 
identity. The others will be subsumed under a new "Teacher 
Center" concept as current funding commitments are fulfilled. 
This study will identify and save those valuable lessons learned 
from past experience for the use of all of BEPD in its future role. 

It is likely that Teacher Corps will be managed from a 
different Federal office in FY 1973. In addition, Teacher Corps, 
a mutable program, changes its characteristics and objectives 
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with each new two'^Veai: operating cycle. Rather than a distinc- 
tive, unified pro^^aKi^ Teacher Corps is like a tandem series of 
programs r one groV^i^ng O^t of another, that are administered by 
the same Bureau, 't'hUs/ I'eachei' Corps will not be singled out for 
individual evaliaa'^We treatment as will the Career Opportunities 
Program. 

Career Opportun jt^g p^^gra m (COP) Impact Study 

This stueiy ii^s P^vii. designed to provide data, testing 
three hypotheses : 

e Hypoth^^is 5-- COP has been an effective ool in 

improv^^^ t^^^ educational experience of students in 
the cl^^5rQ(pn\, 

• Hypoth^^is 2- cop has been effective in enhancing the 
care^jT ^oteJ^"^ials of its participants. 

• Hypoth^^is 3 5 COP has engendered change in institutional 
struct^^es . 

In order t^gt these hypotheses, data will be obtained 
from the followir^5 SOu^^es: 

• Studer^'^^ 

• Partic^^^ant^ 

• Insti'i;'"^tion^ 

1 . 0 Students 

The eventi^^l ta^S^t of all educational reforms is the stu- 
dent. Consequents -^V/ impact evaluation must address the effects 
of a program on t^^^ ^i^uc^ents involved. Unfortunately, in most 
cases, cor being ^>^c^ption, such "higher order" impacts occur 
only after change ha^ t^ken place at previous levels, as in 

5 

EKLC 



the performance of the teacher. Since we are dealing with a 
filter effect, only dramatic changes, in student performance 
will be evident in a short time. 

Our evaluation will be limited to students in grades 1-6. 
We have chosen this grade span for several reasons: 

• Most COP programs are in elementary schools; 

• Achievement tests are available which span these grades 
o The effect of educational inputs becomes more heavily 

confounded with past school experiences and extra- 
school factors as the student progresses through school 

Our evaluation will assess the program's impact on stu- 
dents' achievement and behavior. Attitudinal data is also of 
interest. 

1- Achievem ent Tests : Student tests of basic reading and 
mathematics achievement, such as the Metropolitan or 
Stanford, will be used. Criteria for selection of the 
actual test to be used will include minimization of 
cultural bias and testing duration. Of interest here 
will be a comparison of the performance of students 
who have had COP aides as a part of their classroom 
experience with those who have not. Further, we may 
make comparisons within the sample of COP students to 
investigate possible determinants of impact, such as 
the relationship of the amount of time the aide is 
actually teaching the class with the impact on achieve- 
ment. 

2. Attitudinals: Measures of student's attitudes towards 
school, self, and others will be developed to assess 
differences in attitude between children who have 
COP aides and those who do not. 



2 . 0 Participants 



Although the impact on students is a concern of COP, the 
effect of the program on the aides themselves is of at least 
equal importance. If the program is unsuccessful in training 
aides, no impact on students could be expected. Similarly, even 
if no impact is detected on student achievement, the establish- 
ment of an enhanced set of reality-based aspirations on the part 
of the participants would constitute a degree of success. It 
must be recognized that COP has as an expressed goal, 

...to attract capable persons to careers in education in 
a way that will improve both educat i on and employmen t 
opportunities for the poor , and establish career lattices 
in schools so that productive careers can be followed by 
those recruited through this program. Project Directors 
Handbook , COP Leadership Training Institute, December, 
1970, p.l. 

To explore this, participants will be administered the 
following : ■ 

• A standardized test of basic skills; 
o A questionnaire designed to elicit information 
concerning : 

- career plans and aspirations 

attitudes towards the project, students, peers 
and teachers 

Here we will make longitudinal comparisons of the parti- 
cipants, investigating their changes on these dimensions over 
time. This will be accomplished by use of a pretest and a post- 
test. The data gathered may also serve as the basis of a long- 
term longitudinal study of COP" participation. 

There are two dimensions of participant characteristics 
which are of particular interest in this study. The first is 
the differential effects of the length of time in the COP program. 
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We will obtain information on this dimension when we field this 
study and use it in later analysis, i.e., we will divide the parti- 
cipants sampled into first and second year COP participants and 
analyze for differences in output measures between them. 

A similar procedure is suggested to explore the issue of 
amount of use of COP participants as instructional aides. While 
the Project Director's Handbook specifies that "Teacher aides 
must be used as direct participants in the learning-teaching 
process" (p..4), the question has been raised as to whether COP 
aides are, indeed, used as direct instructional aides in the 
classroom, or relegated to clerical jobs. Again, we will obtain 
this information (i.e., functions of aides) when we conduct this 
survey and develop incidence figures. If the data warrants it, 
we can further analyze the outputs on participants as a function 
of amount of direct classroom instructional experience. 

3 - 0 Institutions 

In discussing institutional change, it is necessary to 
distinguish between simple change and impact. As we intend to 
use the term, an institutional impact is a change in an organi- 
zation which has occurred as a result of the introduction of a 
project but was not a direct part of that project. That is, use 
of COP aides in a classroom is not an impact— it is a part of the 
project's operation. On the other hand, introduction of aides 
funded by the LEA into other schools in the district represents 
an impact, an institutional change which can be related to the 
project but is not an integral part of it. 

Generally, . institutional impact takes time to occur. 
Since COP is only two years old, we would expect little impact 
to be evident even in the oldest sites. Institutional impact is 
a relatively low priority item for COP; however, a multiplier or 
ripple effect would be a welcome finding. To assess the extent 
of institutional impact arising from COP, we will gather information 
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from the following sources: 

1. Teachers in COP schools : We will consider the impact 
of the introduction of COP aides into classroom on 
teachers' behavior and the structure of the class- 
rooms • 

2. SEA: We will survey all 50 SEA's to determine what 
changes in certification requirements have occurred 
in the past few years and if any of it can be attri- 
buted to COP projects in the state. 

3. LEA: We will survey administrators of the LEA (super- 
intendents, principals, etc.) to assess the changes 
in career structures, hiring practices and pay scales 
which have occurred since the introduction of COP. 

4. IHE: We will investigate such things as changes in 
curriculum and faculty make-up, and recruitment and 
entrance procedures. 

5. S chools : We will explore such things as changes in 
schedules and differential staffing. 

In order to make these assessments, the following research 
design will be implemented: 
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Th3 multi'-stage sampling plan outlined above was developed 
to allow us to focus on those populations of most concern, namely, 
COP participants and their students. Use of a probability pro- 
portionate to size (PPS) sample gives each participant in the 
population an equal chance of selection. The procedures described 
below permit us to use, the resultant data on participants directly; 
that is, we can deal with unweighted data at the participant 
level. Results on participants can be generalized directly to 
the participant population. 

Students in Title I schools are primarily from the lower 
end of the socio-economic scale. We know that socio-economic 
index is a potent variable in student performance. Since our 
population is drawn from a narrow range of that scale, we can 
safely assume that variability of performance has been sharply 
reduced, i.e,, we have a very homogenous group. Another way of 
saying this is that the variability within any one class will be 
greater than the variability between classes and schools. 

Procedural Details of the Research Design for the COP Survey 

I, The Site Sample will consist of 16 COP schools selected 

at random with probability of selection proportional to number 
of COP participant. This is a simple one-stage cluster sample. 
It is self-weighting with respect to inferences about the total 
population of COP participants; that is, the simple (unweighted) 
average of the means of the sampled schools with respect to any 
participant characteristic is an unbiased estimator of the overall 
COP population mean with respect to the characteristic. 

The great majority of COP projects have between 30 and 
200 participants each. The exceptions are either so large as 
to swamp the analysis or so small as to have no appreciable effect 
on the results; we shall therefore restrict the survey to this 
size range. Within this middle range, we anticipate that the 
characteristics of principal interest will vary much less from 
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site to site thp ni participant to participant within a site, 

from grade to -adr .r from pupil to pupil within a classroom 
group. 

We originally proposed a somple of 14 sites out of the 
total of 132, which would have yielded estimates of parameters 
at the site level with 95%-conf idence error tolerances of 0.5 
standard deviations: probably sufficient precision, given our 
reduced, relatively homogeneous site universe. The sample of 15 
sites that we now envisage will provide somewhat more precision 
inasmuch as the population from which the sample will be taken will 
be somewhat smaller through exclusion of extreme cases. 

The Part icipant Sample from the selected sites will be 
stratified and balanced so as to insure maximum precision of 
comparison from site to site, from participant to participant, and 
from grade to grade. Within each of the 15 sites, 5 participants 
will be selected at random from each of the 6 grades (1-6) for a 
total of 480 participc^nts. This participant sample will not 
be self -weighting for inferences about the total participant uni- 
verse. If the sampled sites include 1000 participants, this doubly 
stratified sample of 480 will yield a great deal of precision for 
generalizations to the population of participants in sampled sites: 
an error tolerance of less than 0.06 standard deviations, with 
95% confidence. 



The Student Sample will be a two-stage cluster sample of the 
universe of students of COP participants. Within each of the 
first five sites selected in heading I above, we shall test all 
the students of the first two participants selected out of each 
grade. We shall thus have, on the average, 30 students in each 
of 2 classes in each of 6 grades in each of 5 schools for a total 
of 1800 students. This design assumes that pupils within classes 
will vary more than classes within grades or schools within the 
total school population with respect to the characteristics of 
interest . 
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A control group is required for tKe student sample. We 
propoF^'-^ "^r 2 ESEA Title I schools within geographic proxi-- 

mity w ..ools in the subsample. These constraints maxi- 

mize the comparability of the two groups by requiring that the 
schools be eligible for Title I and by adding a geographic con- 
straint. This is not a formal matched group^ as our purpose. is 
to compare two groups of students and not schools. Although we 
have chosen only two sites from which to draw students, we have 
a sample of 3 60 — quite large enough to make meaningful comparisons. 
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II. 

1.0 



INSTRUMENT DEVELOPMENT 

Justificati on Statement for Fall Survey COMB # 51S71034) 



A. 



Justification of Fornn or Other Document in its Relation to O perating 
orResearch Programs =— ^ 



1. The Office of Education, through the Office of Program Planning and 

Evaluation (OPPE), has awarded to Abt As sociates Inc. a contract to conduct 
an impact evaluation of the Bureau of Education Personnel Development 
(BEPD). The study is composed of two sections--a series of project case 
studies and an analytic survey of the impact of the Career Opportunities Pro- 
gram (COP). The case studies are concerned with the impacts of the varying 
strategies used by programs within BEPD and the COP evaluation is concerned 
with the impact of COP on various target groups. Since the instruments sub- 
mitted at this point deal with the COP evaluation only, we will limit our dis- 
cussion to them. 

V/e may summarize our survey efforts in the following very general 
hypotheses, stated positively: 

1. COP has been effective in enhancing the career potentials 
of its pa ticipants. 

2. COP has been an effective tool in improving the educational 
experiences of students in the classroom. 

3. COP has engendered change in institutional structures. 

The documents presented represent only a part of the total COP survey 
efforts. They consist of instruments for Fall administration to participants 
and a questionnaire for administration to the certification officers of the 50 
State Education Agencies fSEAs). The participant instruments include the . 
Adult Basic Learning Examination (ABLE), Level III (Reading and Mathe- 
matical problem sub-tests only), published by Harcourt Brace Jananovich, 
Inc. . and an informal questionnaire developed by Abt Associates Inc. Later 
submissions will contain instruments relating to the survey of student and 
institutional impact. Spring instruments for participants, foi post- test data 
and new informational questionnaires. Case study instruments will be sub- 
mitted for the BEPD program evaluation. 
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The instrument to test basic skills of participants, ABLE, Level III, 
has been developed to cover Grades 9-12, It attempts to measure basic skills 
required for high school equivalency performance. 

In writing materials for ABLE III, the authors made every effort 
to present content and language relevantto the experiences of the 
poorly-educated adult. • • In late winter and early spring 1970, 
ABLE III, Form A, was standardized for school equivalency by 
equating with appropriate subtests of Stanford Achievement Test: 
High School Battery, Communities in Florida, Massachusetts, 
and Minnesota participated in this phase of the research. These 
communities were chosen in order to provide a sample Vv'-ith a 
wide range of ability, socio-economic background, and geogra- 
phic location-. , • The final forms of ABLE III, Forms A and B, 
were also administered, along with appropriate subtests of the 
Stanford High School Battery, Form W, to over 800 adults in a 
variety of academic and vocational education programs across 
the country. 

We are interested in seeing if participants who are starting the second 
year of the COP prograni {and who therefore have had a year of college ex- 
perience) perform better than those just entering the COP program. If such 
a difference exists, we would anticipate a narrowing of that difference in the 
spring (after the new participants have had a year of college). 

In order to reduce anxiety about confidentiality of data, we are asking 
respondents to record their date of birth, rather than their name. By in- 
cluding the year of birth, as well as day and month, the chances of identical 
dates in any site are greatly reduced. Respondents will then be identified by 
site number and date of birth. We need this information in order to match 
up pre- and post-test scores. 

Although impact on students is OOP's primary concern, the effects 
of the program on the aides themselves is of nearly equal importance, A 
goal of COP is to increase the future career potential of its participants. 
This is done by providing occupational training as paraprofessionals and by 
providing college experience- In addressing the impact on these participants 
we are interested in the effects of their training on their grasp of basic 
skills, as well as on their career aspirations. The former impact is evaluated 
by the two subtests of a standardized test and the latter is addressed by 



Karlsen, B, ; Madden, R. Gardner, E, F, , ABLE Handbook, Level III 
Harcourt Brace Jananovich, Inc, , 1971 (pp, 29-30). 
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sections of the participants* questionnaire (items 12 and 13). The rest of 
the Fall questionnaire collects demographic information need ^ foT info- ^ \,i 
analysis. Our Spring testing will investigate the attitudes anci o^.^iiono jf 
COP participants toward education and education- related concepts. Taken 
together, these attributes of the participants outline what may be called 
^^career potential, a collection of skills and opinions which may be associated 
with success in educational careers. 

The impact of COP on a specific institution, the SEA, will be assessed 
by surveying the 50 state certification officers. They will be asked the nature 
of recently instituted or planned changes in state certification requirements 
and if these accommodate the training or experience given by COP. 

The data collected in all phases of this study will be used by Abt 
Associates Inc. in preparing an impact evaluation of the programs of BEPD 
for submission to the Office of Education on August 31, 1972^ 

2» Data concerning participants will be collected on two occa sion s - - the 

Fall of 1971 and the Spring of 1972. This constitutes in part of pre- test/post- 
test design to be used to investigate the changes which occur in the participants 
over time. 

B . Justification of Method Used in Selecting and Contacting Those to be 

Covered 

1. The participant portion of the survey will be used to evaluate the 

impact of the COP experience on the teacher aides involved as participants. 
The instruments will be administered to 480 aides in 1 6 COP sites through- 
out the country. Selection of the participants to be tested will require a 
two- stage sampling. 

Initially, l6 sites will be selected from a reduced population of 90 
COP projects funded in FY 1972. ^ The great majority of COP projects have 
between 30 and 200 participants each. The exceptions are either so large 
as to swamp the analysis or so small as to have no appreciable effect on the 
results; we shall therefore restrict the survey to projects within this size 
range. Within this middle range, we anticipate that the characteristics of 
principla interest will very substantially less among sites than among par- 
ticipants within a site, among grades, or among pupils within a classroom 



1. See Addendum to Supporting Statement for added sites. 



group. We originally proposed a sample of 1 -^fo- r> ^ of the toi I ui i 52: 
the sample of 1 6 sites that we now viivisage will provide somewhat more 
precision inasmuch as the population from which the sample will be taken 
will be both smaller and mere homogeneous. 

The l6 sites "^nl\ be selected at random for the 90 sites reduced pop- 
ulation with the probability cf selection proportional to project size as measured 
by number of participants. Within each of the l6 sampled sites, 30 participants 
will be chosen, 1 5 in their first year in COP and 1 5 in their second, for a total 
participant sample of 480. Selection will be made by simple random sampling 
since the task of developing 16 frames in a size range of 30-200 is manageable 
and a random sample is the most direct approach. 

A selected ite may occasionally have fewer than 15 first or second 
year participants. If this is the case, we ^-vill sample all of the smaller 
group and sample mor- from the other to result in a total of 30. That is, if 
the.e are only 13 fixsi-vear Tiarticipants in a site, we wi?l select all of those 
13 r.nd also 17 second- voar participants. This will not affect the overall 
estimates greatly, as a 100% sample introduces no variance due to sampling 
error for estimates of ^ parameter for that particular site and stratum. 

Our sampling procedure has been chosen in order to accommodate 
expected data analysis. We seek to estimate proportions, mean scores, 
differences among uncorrela:ed proportions and mean score differences 
among groups. We wish to g 3neralize probabilistically both to the total 
population of about 6300 participants in the 90 projects and to the population 
of about 1 350 participants in ^he l6 sampled sites. For generalizations to 
the larger population, our participant sample is a two-stage sample, with 
clustering in the first stage and stratified random sampling in the second. 
For generalizations to Ihe smaller population, it is a single stratified random 
sample. Clearly, the fine structure of the data will be most easily discerned 
in the larter category of gen'- ralizations. 

For a simple estimate of a proportior. p in the l6 site population, for 
example, the half-wi^ith d of 95"^o confide^ ce interval is given conserva- 
tively by ^1 f y \ where n = 480 is the sample site and 



N = 1350 is the populat :>n site. Thus, . . 7 Jp(l-p1. 
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For typical proportions, therefore, the confidence intervals are roughly as 
follows: 

^ Intervals 



0. 1 0. 08 to 0. 12 

0. 5 0. 465 to 0, 535 

0. 9 0. 88 to 0. 92 

We shall estimate population nneans of raw scores on the Adult Basic 
Learning Examination subtests, whose standard devisions range around 7 
or 9. The confidence interval formula here takes the following form: 

d = 2 cr /Yn ^ 1 

We omit the finite population correction as a way of compensating roughly for 
error of measurement inherent in the test. Since these tests have published 
reliabilities around . 93 to , 95 for an adult group with characteristics similar 
to the sample population, this adjustment is probably conservative. As an 
initial crude estimate, then, we anticipate that our estimates of overall 16 
site population score means will be accurate within about 1 raw score point. 

Generalizations to the large (90 project) population will be less 
precise, of course. Let us suppose that the interclass correlations of the 
variables of interest are positive and non-negligible (say, around 0.2 or so). 
That is, the COP sites are fairly similar with respect to the characteristics 
of interest, but there is still substantial variance among the site means. 
Cluster sampling is therefore a relatively inefficient approach and our 
cluster sample of 480 participants might be equivalent to a simple random 
sample of only 100 or so. This would correspond roughly to a doubling of 
the width of each confidence interval above. 

Similarly, comparisons \yithin the population of participants imply 
both smaller samples and also difference estimates. We lost precision in 
both ways. For example, we may wish to compare first-year participants 
with second- year. The confidence interval is inversely proportional to the 
square root of the sample size; reducing the size by half therefore increases 
the interval by a factor of-/2 (about 40%). Further, an estimate of a 
difference also has a confidence interval of about 40% larger than that of a 
simple parameter -stimate in the same population. When combined, these 
effects result in an overall doubling of the width of the confidence interval. 
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The i^^A si>rvey will investigate the impact of COP on the certification 
requirement^ in ty^^ 50 states. Since the population is small and the instru- 
ment is brie^* sill 5^ SEAs will be surveyed. 

Cont^^t Wit^ the sites and Project Directors will be made through the- 
Office of Ed^^ati^j^ and CoP officials. The Project Directors will then 
arrange foj. ^^nt^^t \vith the participants. An oversampling of sites will be 
made to allc?'^ fo:^;. j^on- cooperation which is possible due to the existence of 
other evalxia^^^ns COP being simultaneously conducted. Similarly, an 
oversampUi)^ of participants will be conducted to allow for those who cannot 
attend the te^^ing. 

Initio coi^(;^ct with the SEA certification officers will be made by a 
letter frorri A^t A^^^ciates Inc. asking for their cooperation in the survey. 
Interviewing ^onta^^ ^;^dll be made by a telephone call from the Abt Associates 
staff mernbe^ co^^^ucting J:he interview. No difficulties concerning non- 
respondence ^l* Va.li<3ity are anticipated. 

C. Bji^i^^ej^jli ^tio n of Plans for Collection, Tabulation and Publication 

1. Fof i'^^ p^:j:'^icipants, the pre-test data collection will be conducted in 

each of the site^ in the Fall of 1971, beginning about October 19 and ending 
about Noven^^^r 5^ About one day will be spent at each project. The post- 
test phase t^onducted in the Spring of 1972, beginning about March 13 
and ending a^^ut }Jl^y 5. The exact amount of time spent at each site will 
vary accofdi'^S to amount of other data to be collected, although the col- 
lection of th^ ^3^t^ ^^lated to the instruments submitted at this time will 
require app^j'^.^in^^.tely one man-day per site. The testing will be conducted 
by membe^.^ t}^^ Abt Associates professional staff in cooperation with the 
project st^ff/ 

SEA ^^ta wiU be Collected by telephone interviews in late October 
and early Isjc^'^^mb^r » l97l. The entire period should encompass two or 
three weeks^ Altli^^^gh the actual interview is brief, some difficulty in 
contacting a Usy 0^^te official is likely. The interviews will be conducted 
from the Ab^ Asso<^^^tes offices in Cambridge, Massachusetts by staff 
members. 
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2. Participant results will be tabulated on a pre- /post- test by partici- 
pant basis. Participants will be further categorized on the basis of certain 
items in the questionnaire (such as items 3-10) which gather demographic 
data for use in later analyses. Other questionnaire items will be tabulated 
(by participants) over time to allow investigation of change in response (such 
as items 12-13). Here, the dependent variable for analysis will be the 
presence or absence or change over time. Only the total scores on the stan- 
dardized tests will be recorded, not individual item responses. 

Subsequent analysis will attempt to document the existence of changes 
in participants in COP projects over time and to associate certain character- 
istics of the participant or his project (gathered in the questionnaire or from 
other sources) with the magnitude of change. Techniques used will include 
parametric and non- parametric tests of central tendency, cross-tabs correla- 
tion and possible regression analysis. 

Items in the SEA questionnaire will be tabulated separately on a state 
by state basis* Analysis will assess the total amount of impact found nation- 
wide with some attempt to determine the extent to which COP projects directly 
influenced the changes. 

3. The data collected will be presented in the impact evaluation report 
to be submitted to the Office of Education on August 31, 1972. 

D* Documentation of Consultation with Those Supplying Data, Users of 

the Data, and Others 

1. The standardized tests were developed and validated by the professional 

staff of their respective testing firms. Data concerning specific aspects of. 
test reliability, validity and other measurement concerns can be found in the 
test manuals. The COP participant questionnaire and the SEA interview for- 
mat were developed primarily by Dr. Sydelle Shapiro and John Doucette of 
Abt Associates Inc. with the assistance of other staff members. 

Selection of the standardized tests was made in consultation with 
Dr. Robert Hall of OPPE, Dr. Wilton Anderson, Director of COP, and his 
staff. The content of the two questionnaires was discussed with Dr. Anderson 
and other COP staff members. 
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2. The estimated time per testing session for COP participants is two 

and one half hours, each session, involving 30 participants at each o£ the 
16 sites. The burden on participants, then, is estimated to be 1200 man- 
hours per testing. For both the pre-testing and post-testing the total burden 
would be 2400 man-hours. 

The estimated time of the SEA interview is ten minutes. The total 
burden on the state certification officers is therefore estimated at 12-14 
man-hours. 
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2.0- FALL SURVEY INSTRUMENTS 



Questionnaires 

1. COP Participants Fall Survey 

2 . SEA Telephone Questionnaire 



Standardized Tests 

Adult Basic Learning Examination (ABLE) * 
Math and Reading Subtests, Level III, Form A 
Harcourt, Brace and World, Inc. 
7 57 Third Avenue 
New York, New York 



* For a specimen ABLE test booklet, please refer to the 
Spring Field Guide, P. 
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. 0MB Number 51S71034 
Approval Expires Aug. 1972. 



(1-5) 
(7-8) 
(9-12) 



Please do not write in this box; it is for coding use only 

IMPED 

W 



CAREER OPPORTUNITIES PROGRAM : Participant Fall Survey 



We would appreciate your help in our study of the Career Oppor- 
tunities Program, and would like you to fill out this questionnaire. 
Your answers will be used only in this study and will be kept confi- 
dential. Please do not write your name on this form; instead, 
please enter the month, date, and year of your birth in the blanks 
below. 



Date of Birth: 



(14-19) 



(month) 



(date) 



(year) 
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ir 

1 

2 
1 

2 
1 

2 
3 
4 
5 
6 

1 

2 
3 

1 

2 
3 
4 



pahTicipant questionnaire 

TIlis questionnaire is being administered by Abt Associates ns part of a research 
project for the Office of Education, Washington, D.C. 

Please help us! 

We would like to ask you a few questions to help us in describing COP. Only you 
know the answers to al! these questions. They will be used only for the purposes 
of this study. Please put a check [f^ in the bracket that best answers each of 
tl'iese questions. 

Question 1: Were you a teacher aide before you joined COP ? 
[ ] No 
f 1 Yes 

Question 2: Is this your first or second year in the COP program? 
[ ] First Year 
[ ) Second Year 

Question 3: Before joining COP, how far had you yourself gone in school? Please 
check the box which includos the grade you finished. 

[ ] Grades 1 - 3 

[ ] Grades 4 - 6 

[ ] Grades 7 - 8 

[ J Grades 9-11 

[ ] Grade 12 (high school diploma) 

[ ] Some college or more 

Question 4: Before joining COP, was your schooling in the United States or was 
some of it outside of the United States (for examjJe, Mexico)? 

[ ] All of my schooling was in, the U.S. 
I ] Some of my schooling wasMn the U.S. 
[ ] None of my schooling was in the U.S. 

Question 5: What language is your first language (that is, the one you learned 
at home)? 

I 1 English 

[ ] Spanish 

[ ] American Indian 

[ ] Other (please specify) 
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(25) 



(2G) 



(27) 



(28-29) 

(30-31) 
(32) 



(33) 
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1 

2 

1 

2 
3 
4 
5 
6 



2 
3 
4 
5 



1 

2 

.3. 

4 

5 
6 
7 
8 



Question 6: Please indicate your sex. 
] iVIale 
] Female 

Question 7: Which age group are you in? 

J Under 20 

] 20-24 

] 25-29 

] 30-34 

] 35-39 

] 40 and over 

Question 8: What is your marital status? 



] 



] 



Single 

Married 

Widowed 

Separated 

Divorced 



Question 9: How many children do you have? 
(write number of children on blank) 

Question 10: How many children do you have who are. younger than six 
years old? 

(write number of children under six) 

Question 11: Please indicate which ethnic/cultural group you belong to 

[ ] American Indian 

[ ] Black 

Oriental ••.v.. ■ ,. ,. 

Spanish : 
Mexican 

pLJerioRican 

Other 

White (other than Spanish) 

Other (specify) , 



I :■] 

I ] 

I ] 

[ ] 

[ ] 

( ] 



Question 12: What did you do last before joining COP? (specify) 

[ ] Teacher Aide 

f ] Housewife 

( ] Armed Services 

! ] Unemployed 

[ ] Other (specify) 
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(34) 

(35) 
(36) 
(37) 
(38) 
(39) 
(40) 
(41) 
(42) 



Question 13; What kind of work do you plan to do next year? (you 
may check more than one,) 

Go out of the teaching field entirely 

Become a full-time tear er 

Go into other Federal \ grams 

Become a full-time hOLGewife 

Go into the Armed Services 

Continue in the COP program 

Become a full-time student 

Leave COP and become a teacher's aide 

Other (specify) 

Don't Know 



] 



] 



Question 14: What are your career plans over the next three years? (you 
may check more than one.) 



(43) 


1 


[ ] 


Go out of the teaching field entirely 




2 


[ ] 


Become a full-time teacher 


(44) 


1 


[ ] 


Go into other Federal Programs 


(45) 


1 


[ 1 


Become a full-time housewife 


MR) 


1 
1 


[ 1 


Go into the Armed Services 


(47) 


1 
1 


[ ] 


Continue in the COP program 


(48) 


1 


[ ] 


Become a full-time student 


(49) 


1 


{ 1 


Leave COP and become a teacher's aide 


(50) 


1 


[ ] 


Other (specify) 


(51) 


1 


[ ] 


Don't Know 


(52) 


2 


1 1 C 

1 J Q For Coding Only 






Question 


15: How did you first hear about COP? 


(53) 


1 


[ ] 


Friend ' ■ • .•- ■ • • v ■ ■ ■.. 




2 


[ ] 


Teacher 




3 


.[ ] 


Community Organization; which one? 




4 


[ ] 


Newspaper 




5 


[ ] 


Radio or TV 




6 


[ ] 


Other Federal or State Program 




7 


( ] 


Other (snecifvl 






Question 16: Did you experience any difficulties in apply ng for or entering 
the COP grogram? 


(54) 


1 


[ ] 


No 




2 


[ ] 


Yes; What were they? 










(55-56) 
(57-58) 




RS 

For Coding Only 


O 
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Development of Student Qpinionnaire Instrument 



Although generalized measures of attitudes are quite common, few 
attitude instruments appropriate for use in group testing of yoijing children 
are generally available. Several self-concept instruments are used widely, 
however, they are much too narrow and time consuming for our purposes. 
Since a group administered test of affective impact of COP on students is 
desirable, we decided to develop an instrument for this purpose. This 
instrument would be used as an alternative to a questionnaire; that 
is, we would use it as a survey instrument. 

We could not adapt the typical adult attitude scale for sev- 
eral reasons. First and most important, these require a degree of 
verbal skill that cannot be expected of young children. Even if 
the standard instruments were read aloud to the students, they 
would not likely understand some of the words, and changes in the 
wording would alter the actual content of the scale. In 
addition, the response modes are complex. Rankings are clearly too difficult. 
Agree/disagree responses introduce an acquiescence effect. It is not un- 
reaisonable to assume that children tend to acquiescence and will therefore 
agree more than disagree, regardless of the content of the items. Further, 
commercially available scales are generally ^'trait measures" using a large 
number of items to measure a single underlying trait. Since we were in- 
terested in studying attitudes not as traies but as opinions, such 
scales are inappropriate. 

If the items are designed to be administered by being read aloud and 
the language is intended to be understandable to young children, the initial 
objections can be overcome. Market researchers have used a type of Likert 
scale with children with some success. In it, the options on the scale are 
represented by a series of faces with expressions changing from smiling to 
frowning.- It has been demonstrated that children are able to respond appro- 
priately in this mode. Finally, savings in time can be made by using the 
opinionnaire method rather than the trait measure approach. For example, 
if we wish to know a student's attitude towards his teacher , we ask him to 



See Appendix "Pretests of Student Opinionnaire Instrument" 
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mark the face most like his feelings toward the teacher. Marking a smile implies 
a positive attitude, marking a frown implies a negative one. Since this instrument 
will not be used for individual diagnostic purposes, such general measures have 
•■•ufficient reliability if group rather than individual data is used. 

Our goal then was to develop an opinionnaire appropriate for use 
with children in grades 1-6. It would require no reading on the part 
of the students, since it would be administered verbally, and would 
have a simple response mode. This experimental instrument would not 
be suitable for clinical purposes, but would be appropriate for re- 
cjftarch involving groups of students. In essence, then, we developed 
a survey instrument to Treasure the opinions of children, not their 
underlyiny -crai^cs • 

Discrimination Test of Student Opinionnaire Instrument 

In order to determine whether or not first grade students could dis- 
tinguish '-'etween the five faces (sequenced from happy to sad), a field test 
was conducted at a school in Cambridge, Massachusetts. The Russell School 
was chosen because of its location in the inner-city and the fact that it draws 
students from the surrounding low-income neighborhood. Although the neigh- 
borhood has its own unique characteristics, it includes a racial/ ethnic and 
S.ES mix similar to other low-income neighborhoods across the country. 
Therefore, the students at the school are fairly typical of the types of students 
found in inner-city schools, including those in which COP participants are 
placed for in-service training. 

O i**: of a first grade class of thirty children, five children were cho- 
sen by Li.e teacher to be tested. The five children were chosen on the basis 
of being "average in ability" and havmg little fear of strangers. 

The test procedures are described in detail in this riSDOrt. 
All five children were able to discriminate between the 5 faCRf=^ 
and to respond to the examiner^s questions ("Which face looks very happy? 
"Which face looks almost happy?", etc. ) While all children could discriminate 
the faces, none of the children could sequence the face's themselves (either from 
happiest to saddest, or from saddest to happiest) without identification by the exam- 
iner as to which face followed which. All the children were able to sequence the 
faces with the examiner's help. Sequencing is not a skill necessary for this in- 
Q strument; discrimination is. 
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On the basis of this test we felt certain the nnost, if not all, first grade 
students would be able to discriminate between the five faces and make identi- 
fications according to happy/ sad characteristics, given appropriate instructions, 

Instrument Pre-Testing 

A twenty minute test was developed for pre-testing. The following 
questions were addressed in the pre-test: 

• Can children discriminate between stimuli? 

• Will children show a spread of responses or will they 
only alternate or perseverate? 

• Is the spread of responses a random effect? 

• Will children repeat their choice at the beginning and 
at the end of the test? 

The first pre-test was conducted in the first grade of the Russell 
School,, where the original discrimination pre-test was conducted. Revisions 
on the original test (Pretest Procedures for Student Opinionnaire Instrument! 1] 
following) were made on both the procedure and times used, as a result 
of this pre test. The revised Pre-Test Procedure for Student opinionnaire 
Instrument (Z) was then tested in a third grade class at the school and sub- 
sequent changes were made only in the procedure (see Revised Pre-Test 
[3]. Arrangements have been made for further testing of the instruments in 
Newton, Massachusetts, Port Edwards, Wisconsin, and Manchester, 
New Hampshire. 

Items 

Entries in the list of stimuli given to sutdents fall into two categories: 

• A f ew items, placed at the beginning of the list, will be 
selected to elicit strong positive .d negative responses, 
thereby familiarizing the students with the scale they are 
expected to use in marking their answers. 

• The remainder of the items will be selected from a list 
which is now being prepared. They will give direct be- 
havioral measures of student opinions about themselves 
and their classroom environments. These items will be 
selected on the basis of their face validity ( clear relation 
to the behaviors being measured) and the range of 
response. Items on which students express a wide 
range of views will be preferred to those in which most 
of the respondents cluster at one end of the scale. 
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Expectations for Use 

We have developed this instrument in order to explore attitudes to- 
wards school, self, and others among grade school children (grades I-VI), 
Of specific interest is differences between those classes of children who 
have had a COP aide in their class and those who have not. This instrument 
was developed as a way of measuring group ^ as opposed to individual, 
response. For example, we may choose to compare the average response 
on an item (e. g, , opinion of "spelling") for the group of students having COP 
aides with the average for a group not having them. 

What research uses beyond the present study could the instrument 
have? It is important to re -emphasize that it cannot be used as a diagnostic 
instrument for individual subjects. Neither is it likely to have sufficient 
reliability for use in measuring change.' However, although the items 
were selected for their face validity specific to our needs, it 
could be used^ if it is successful and possibly with modifications 
in the content of the items, in other research and descriptive 
studies of this population. 
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PRETESTS OF STUDENT QPINIONNAIRE INSTRUMENT 
Procedure for Discrimination Test of Student Qpinionnaire Instrument 

1. Ask the teacher to choose one child who: 

• is of fairly average ability commensurate with his 
age group 

• is fairly out-going r.nd would be willing to. classify 
verbal meanings and facial expressions. 

2. Meet the child and talk with him for a moment — get ac- 
quainted for a little while. Explain that you would 
like to play a little game with him. Show him the pic- 
tures of the faces and ask him to: 

a. Identify the HAPPIEST face in the group; 

b. IDENTIFY THE NEXT HAPPIEST face ("Which face V70uld 
be the happiest one if I took this one away?") ; 

c. Identify the SADDEST face in the group; 

d. Identify the NEXT SADDEST face CWhich face would 
be the saddest one if I took this one away?"); 

e. Identify the face that doesn't seem to be feeling 
ANYTHING — that is, the face which doesn't look either 
happy or sad. 

3„ Ask the Child if he could arrange the faces from saddest to 
happiest. (This may be extremely difficult for children 
to do--so don't press them if they can't do it.) 



^•^^ Revised Student Opinionnaire Procedures 
first Pilot Tes t 

' — ■ , • - .-j/^.,.^ j ga SL-L Ul)iLU 1 u ( J ; 

"Here is a -:c(urr. of faces. Some facer arc happy; so-ne faces are 
i-niahpi^y. Here is happy face; here j .s another happy face. V/hich face is 
the happiest? Shovr me a face that is unhappy. Show me another face that 
is ur.}-,appy. Which face is the most imhappy? 

What is thi-o face? It is a face that is neither happy nor unhappy. It 
is in-between! 

0!:ay, onrc xiiore. What kind of face is this? AjkI {Vyir-o /_,..,] t;.-.; ■• 
( etc. , etc. ) 

Good. Now you know what each face means. Now Ihis is wha1: wo 
are ^oing to do. I cm going to say diiferent things. Each time I say 3c.:..c- 
thing. you v.ill sho-v me which face goes with what I say. Suppose I said.. 

'oraiiuo Juice. ' Ivovv if oran,'j;e -juice made you snd or nnhav.^-. 

would choose this fa ce. If orange juice makes you very, very happy you 
would clioosc til is face. 

You can shov me which face tells how you feel by drawing a lir-e 

t]iroii;;h it. 

I- think you v ill find this fun to do. Let's try it, okay? 
The first set of £accs is next to the letter 'A'. Draw a circle around the 
face that tells best }-,o\v you feel when I say 'ice cream' (A). Okay, now 
the next set of faces is next to 'B', and tlie word is 'running' (B). " 
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Itern List (1) 


C. 


Going to Bed 


D. 


Having lunch 


E. 


Playing kickball 


F. 


Pv^eading a booh 


G. 


SLn:J, Ing a song 


H. 


Medicine 




T.V. 


.T. 


Spelling 


K. 


Vj:.5 5 (teacher) 


L. 


If your teaclicr is sick and slays out of .school 




How do you fee] about yourself? 


N. 


Scliool 


O. 


Vacation 


P. 


Arithmetic 


Q. 


Mailman 


R. 


Spina CM 


S. 


K you arc sick and have to stay home from schoo] 


J. , 


Candy 



I 

\ 
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Second Pilot Test . . 

Revised Pre-Test Procedure for Student Opinionnaire Instrument 

Required Steps and Procedures 

K Test booklets are not handed out until description of test and initial 
instructions are given, 

2, Students must be lined up in rows* 

3. Teachers are to- circulate during the first page but then must move to a 
far corner of the room. 



»'Here is a picture of faces. Some faces are happy; some faces are 
sad. Here is a happy face and here is another happy face. Which face is the 
happiest? 

Show me a face that is sad. Show me another face that is sad. Which 
face is the saddest? 

What is this face? It is a face that is neither happy or sad. It's 
in-between! Okay, once more--what kind of face is this? This? This? 
[etc.] 

Good. Now you know what each face means. This is what we are 
going to do. Pretty soon I am going to hand out some booklets that have 
these faces in Ihem. They look like this. • . [hold one up]. . . This is what I 
want you to do. I will read a list of things to you. When I read one, I want 
you to draw a line through the face that shows best how you feel. Suppose I 
said, 'taking a bath'. . . [point each time and draw a line through the appro- 
priate face]. . . Some of you like a bath a lot or a litte. Some of you don't 
. - care. Some of you hate taking a bath a lot o_r a little. 

, [Hand out papers]. . ..When you get your papers, please do not write 
your name. What we do want to know is where you are a boy or a girl. 
Write a G if you are a girl. V/rite a B if you are a boy. Put the letter here. . . 
[point]. next to Page 1, Do that now. All done? 

There is one thing that's very important. We only want to know how 
you yourself think. There are no wrong or ripht answers. Your ideas are 
as good as anyone else's. The only good answer is the one that you iliink 
is right* But, we want your own ideas on tho. paper. D on't call it out ! 

I think you will find this fun to do. Let's try it, okay? The first 
set of faces is next to the letter A. Draw a line through the face that tells 
best how you feel when I say, 

'ice cream'. 

Good. Now the next set of faces is next to B and the word is ... " 



Revised Item List (2) 

B. Running 

C. Going to bed 

D. .Having lunch 

E. How do you feel when you look in the mirror? 

F. Reading a book 

G. Singing a song 

H. Medicine 
L T.V. 

J. Your Teacher 

K. Spelling 

L. If your teach is sick and out of school 

M, How do your feel about yourself? 

N. School 

O. Miss (teacher) 

P. Working with numbers 

Q, Mailman 

R, Spinach 

S. If you are sick and have to stay home from school 

T. Candy 
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Tliird Pilot Tost 

Revised Pre-Test Procedure for Student Opinionnaire Instrument (3) 
Rc£Tvu_rod^l.'ns a:-. c: PTor odin-cs 

1. Test boolJci s arc i^oj;^ handed out until doscription of test and initial in- 
strucLio'i!, a r-- >.'.i\ L:ji, 

2. Students ^nn p< bo linrd up in rows. 

3. Teachers; r.rc <o circnb-te durin- the first page but then must 'v.ove In a 
f-Lr corn err oi the room. 

4. Smile at tho c.hUdrcT: and do whatever seems appropriate to rola-: thorn. 

5. Get the cldldrcn to work with you during i.n.structions. 

f->. When rc-..dir-. the lir.t, preface each itein with "How do you feei n bout/\vl,en? " 

7. Give a full 2". -30 scc-cnds per ittm. Doj^'t j -ur.lt ! 

"I thi]ik you will like what we are; goiji;; Lo do today. You'll tell me 
when we're fini?r.,c:d, okay? 

•Here is a picture of faces. Some facoF: are happy; ijome faec;- are 
sad. Here is a h---.ppy fne.c and here is another happy face. Which l;;c:e i.s 
tlie iiappicsv? 

Show me a face that is sad. Show me another face tliat is sad. V-7hich 
face if; i ! 1 1; s £; d d e < t ? 

What is; t!-.:3 face? It is a face that is neither happy nor sad. It is 
in-between! Okay, once more - -v,d,at kind of faee is thi.s? This? This? (etc.) 

Good. N;-.w you know what each face means. Tliis is v/hat we are 
poij^p to do. Pretty soon I am. going to hand out some booklets that have tiiesc^ 
faces in them Ti;ey will look like this. . . (hold one up). . . This is what I want 
you to do. I will read a list of thin-s to you. When I read one, I want you 
to draw a line through the face that shows best how you fee. Suppose I said, 
Makinn a batli'. . . ^joint each time and draw a line throunh the appropriate 
fare). . . Son-,e of you liJ-.e a bath a lot o_r a little. Some of you don't care. 
Some of you hate taking a bath a lot <vr a little. 

(Hand out paper s )... When you p,ct your papers, pl<.-;, se do not v.-rite 
your name. Whal we do want to know i.s whether you arc a girl or a boy. 
Write a G if you are a pirl. Write a B if you are a boy. Pvit the letter he: e. . . 
(point), .next to Pap.e 1. Do that now. All done? 
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There is one thing that is very important. We only want to know 
how you yourself think. There are no wrong or right answers. Your ideas 
are as good as anyone else's. The only good answer is the one that you thin! 
is right. But we want your own ideas on the paper . Don't call it out! 

I think you will find this fun to do. Let's try it, okay? The first set 
of faces is next to the letter 'A'. Draw a line through the face that tells 
best how you feel when I say, 'ice cream' (A). 

Good. Now the next set of faces is next to 'E'. , . 

Revised Item List (3) 

B. Running 

C. Going to bed 

D. Having lunch 

E. How do you feel when you look in the mirror? 

F. Reading a book 

G. Singing a song 

H. Medicine 
L T.V. 

J. Your teacher 

K. Spelling 

L. If your teacher is sick and out of school 

M. How do you feel about yourself? 

N. School 

O. Miss (teacher) 

P, Working with numbers 

Q. Mailman 

R. Spinach 

S. If you are sick and have to stay home from school 

T. Candy 
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Jusrix::! nation Statement for Spring Survey 



A. Justification of Form or Other Document In Its Relation to Research 

Programs 

In September of 1971 Abt Associates Inc. submitted to OMB a request 
for clearance of instruments to be administered in the first of a two part quan- 
titative impact evaluation of the Career Opportunities Program. The request 
was approved (Approval #51571034) and the instruments were subsequently 
used in a fall survey of participants in the Career Opportunities Program, 
The approved supporting statement is attache'd as Appendix A. The present 
request for approval is made within the context of the earlier supporting state- 
ment. It is a continuation of the survey plans which called for revisiting sites 
in the spring and conducting both post-tests, new tests, and additional inter- 
views. The instruments to be presented now are those to be used for the new 
tests and the additional interviews. Accordingly, our discussions will be 
limited to them. 

We may summarize our survey efforts in the following general hy- 
potheses, stated positively: 

1. COP has been effecTn e in enhancing the career potentials of 
its participants. 

2. COP -::s== been an effective tool in improving the educational 
expp-3CT-:es of students in the classroom. 

3. COP :Tas engendered change in institutional structu res. 

The doc nts to be presented represent only those ne- to the spring 
sur--ey efforts, r:.r- instruments will be administered to the following 
grorps: 

• COP lies 

• COP Pr-v.ect Directors 

• Superr .lendents of participating LEA's 

• Administrators of participating IHE's 

• Principals of participating schools 

• Students in classrooms with COP aides (and an associated 
comparison group) 

• Teachers of classrooms with COP aides (and an associated 
comparison group 

(See Matrix on page 15. ) 
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(a) Participants; A goal of COP is to increase the future career potential 

of its participants. This is done by providing occupational training as para- 
professionals and by providing college experience. In addressing the impact 
on these participants we are interested in the effects- of their training on their 
grasp of basic skills, as well as on their career aspirations. Basic skill 
level is evaluated by the two subtests of the Adult Basic Learning Examination 
(ABLE): career aspirations are. addres sed by sections of the participants' 
questionnaire. Our spring testing will also investigate the attitudes and opin- 
ions of COP participants toward education and educcition-related concepts. 
Taken together, these attributes of the participants outline what may be called 
"career potential," a collection of skills and opinions which may be asso- 
ciated with success in educational careers. Participants will be post-tested 
on the Adult Basic Learning Examination, Form B, in order to note any 
changes in basic skills over time and to relate such changes to data or demo- 
graphics and/or opinions and attitudes toward the program. In addition, 
attitudes toward the program will be related to job characteristics and current 
college experiences. 

(b) Students: Impact on students will be measured by achievement test 

scores and student opinions about school, self, and others. Standardized 
achievement test scores (reading and math subtests) supplied by the sampled 
sites will be obtained for students (grades I-VI) with COP aides and compari- 
son students who do not have aides. The question of interepl here is whether 
children in classes with COP aides perform better on standardized tests than 
similar students who do not have aides. In those sarrqiLLed sites where achieve- 
ment testing is not scheduled in the spring, Abt Associates Inc. will test 
using the Cooperative Primary Test (grades I-III) and the Comprehensive 
Test of Basic Skills (grades IV-VI), reading and math subtests. 

It is generally agreed that achievement gains are difficult to measure. 
Furthermore, academic achievement alone has never been the total goal of 
public education. Hopefully, schools help children achieve a strong and 
positive sense of self, others and school. These opinions cannot be measured 
in an achievement test. In order to. assess opinions about self, others and 
school among students in grades I-VI Abt Associates Inc. has developed a 
"Student Opmion Instrument. " ' We will administer this instrument to 
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all students in the sample and comparison classes to see if there are differ- 
ences in attitudes to^^ard school, self and others between children who have 
a COP aide in their classroom and those students who do not. 

(c) Institutions: Institutional impacts will be m_easured by type. Changes 

in classroom, elementary schools, L,EA standards, IHE courses and require- 
ments will be explored, as described below: 



• 



Teachers will be asked to respond to a questionnaire which 
has been designed to enable us to collect data on changes 
in instructional scheduling, changes in extra-classroom 
school-related activities, attitudes towards the Career 
Opportunities Program and selected demographics. Of 
prime interest is the relationship between presence of a 
COP aide and instructional scheduling, and extra -clas s room 
school-related activities, 

• Principals v^ho have COP aides in their schools will be 
asked to giv:- their opinion concerning the impact of COP 
aides, both on classroom activities and elassroom structure. 

• Superintendents will be queried for data on changes in 
hiring policies, requirements, options and opportunities. 
In addition, .ittitudes toward COP aides will be assessed. 

• The instruments for college administratcir s explore 
relationships between COP and changes in courses, ad- 
mission reauirements , degree requireir-ents and admis- 
sion criteria , 

• The brief ij:istrument given to Project Directors will 
obtain information on recruitment procedures, community 
interest in services of COP aides, function and structure 
of advisory councils and some factual information for 
validating purposes. 
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B. 



Justification of Method Used in Selecting and Contacting Those to Be 
Covered 



(a) Pilot Tests: The design for second wave data -collection calls for a 

more elaborate and extensive survey than did the fall testing. There are a 
total of nine distinct instruments which must be administered to seven cate- 
g ories of respondents (see Section A). To facilitate our field work it is neces- 
sary that most of these instruments essentially self -administered. At the 
same time, we wish to guard against the danger of different respondents 
reading different meanings into questions. 

Accordingly, on December 15, 1971 a trained field staff team from 
Abt Associates Inc. administered all the instruments to COP participants and 
related administrators in Manchester, New Hampshire. The Project Director, 
Dr. Cavanaugh, graciously allowed us to pilot test our evaluative instruments. 
All test±ng was conducted at a single school; there were between one (Project 
Director! and nine (participants) respondents for each questionnaire. This 
pretesting of the spring questionnaires was supervised by a field staff memlber. 

(b) Sampling: As previously indicated, several groups of respondents 

will be involved in the COP spring survey. The composition of many of these 
groups was determined by the sampling procedure for selecting participants, 
conducted prior to the fall testing of participants. This sampling plan has 
been fully detailed in a previous justification statement, approved by OMB, 
attached to the present submission, and will therefore not be reviewed here. 

The selection of 16 COP project sites as the first stage of the parti- 
cipant sample determined the sample of COP Project Directors, Superinten- 
dents of participating LEA's, IHE Administrators, and Principals of partici- 
pating schools. All of these individuals in the 16 sites already selected will 
be included in our survey. In cases where thera is more than one individual 
of a given type (as when there are several participating schools), we will in- 
clude all of the appropriate individuals. Generalizations from data so gathered 
to the restricted universe of 90 COP projects have the same precision as the 
first stage of the participant sample, discussed in the previous submission. 
Selection of all possible respondents at a selected site (100% subsampling) 
adds no additional error to the estimate. 
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Contact with the Project Directors has been establisned in arranging 
for the fall testing of participants. Contact with the other pe:r3onnel to be in- 
volved in this part of the spring survev will be established through our liaison 
at the site (usually the Project Director or his deputy). A 200% response rate 
is anticipated, as all respondents will be personally visite£ either by an Abt 
field staff member of by the liaison to collect the completed questionnaire. 

The sample of participants for the spring testing w/ill be those mem- 
bers of the fall sample that attended the testing sessions. Sin-e we will be 
collecting data relating to change in :::B.rticipant characteristics over time, 
data for an individual must be avail^-le at both poiists. (The technical aspects 
of the sampling are discussed in th- devious justification statement attached 
to this submission. ) Initial contact lias been established with the participants. 
They will be contacted again by the piroject liaison when a date for the testing 
at their site has been established. 

The only sampling that is new for the spring is the sample of students 
with COP aides, teachers in classrooms with COP aix:.es an:d a comparison 
g^roup for both. Due to constraints of time and budget, the student and teacher 
samples are somewhat less precise than those for particix>ants or institutions. 
While pre-/post-testing was available to' us for participant testing, 
it was not a realistic option for student testing because {a) extreme sensi- 
tivity of the sites to student testing, and (b) time constraints. 

a. The COP staff expressed great concern that entry into 
the sites would be denied us by immediate requests for 
student testing. They were also concerned leit this 
request compromise the COP position at these sites. 
We simply would not be welcome. 

b. Time constraints strongly argued against this as well. 
Pre-Zpost-testing would require that we arrange for 
student testing as soon as school opened, before the 
students had worked with COP aides. Clearly, the 
LEA^s would not be able to manage this even if they 
were receptive to such testing, what with the start-up 
problems of the school year, such as class assign- 
ments, COP assignments, class scheduling, and our 
simultaneous need to receive lists from thein of classes, 
teachers, and COP aides in order to develop sample 
groups. 
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In the light of these considerations v,^e further constrained. . ..^ con- 
trolled groups, i, e. , the decision was made to test classes ol szii: -ts from 
the same school (for COP aides vs> no aides). Where this proced' 
not be followed (e.g. , all classes had aides; insufficient classer " 
we then drew a sample of non-aided classes from the nearest sc 
district which had students of comparable SES to the students o: tr 
aided classeso 

The student sampling will be a multi-stage sample, 
project sites at the first stage and classrooms with aides at ti-- 
tact classroom units will be used; that is, all students in a sel-c- 
are therefore included in the sample. The first five sites seleicir 
ticipant sampling constitute the first stage of the student sampi.- 
of these sites, a sample stratified by grade of classrooms conta:-- 
aides will be selected. Two classes from grades I-VI will be cir-: 
total of 12 classrooms per site and 60 over the five sites. 

We may fruitfully conceptualize this part of the study 
ment in which there are groups which receive different treatmer-r 
a classic analysis of variance design, i. e, , a nested groups wir^i. 
design. 

Our design may be characterized as a mixed-model design: with sub- 
j'M :.s nested within classes nested within methods, "We begin by conceptual- 
izing a treatment population, differing systematically only with respect to 
the level of the treatment variable. A, From each population, subjects are 
randomly selected in groups of size n; the sampling process ends v. i-en g 
groups of n subjects have been sampled from each population. In this 
total variability amon^ s\ibjects has three potential sources: 

• Treatment Effects: COP aide/no aide 

• Group Effects: grade/class 

o Residual Individual Effects: attitude ability 



could 
grade), 

-.n the 

::dp- 

zg COP 
■i:d. In- 
:£_as s room 
r the par ~ 
Imin each 
COP 
for a 

xperi- 
This is 
reatment 



Myers, J. L, Fundamentals of Experimental Design, Allyn and Bs^con^ 
1966, page 213, 
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The paradigm below defines the design where, 

2 
1 



2 

A, = Treatments (fixed variable) 



Aj^ = COP aide in class 
A^ = No aide in class 

G^^ = Groups (random variable) 



Classes chosen within constraint of two from each 
grade/ site 



Nj^ = Subjects 



Students within each classroom 



^1 



N30 ^30 ^30 ^SO ^30 ^30 



The sources of variance and degrees of freedom are described in the table 
Delow: 

Analysis of variance for groupp -within-treatments design 

SV df_ 

Total 3599 

Between G 119 

A 1 

G/A 118 

S/G/A 3480 
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Conventional F tests will be made using a . 05 level of significance. • 
We may characterize the results as if an experiment had been performed. 
That is, we have used an inferential procedure having the probability o£ a 
Type I error of . 05. 

The sample frame will be developed in conjunction with the Project 
Director. He will identify all classrooms in the sites that have COP aides in 
them and send us a listing of them. From this list we will select a sample 
of 12 classrooms, two each from each of the first six grades, for a total of 
60 classrooms. 

Initial contact with the LEA's involved in the student sample has 
been made. We have been assured of cooperation by the research and evalu- 
ation staff of the five LEA's, We do not anticipate any problems with non- 
response. 

A sample of comparison students will also be drawn from each of 
these lea's. A frame consisting of classrooms of grades I VI without aides 
in schools that, in the opinion of the evaluation staff of the particular LEA, 
are comparable to COP schools in SES characteristics will be developed. 
The nature of the classrooms composing this frame will be determined by 
the lea's in consultation with Abt Associates Inc. From the frame we will 
select a sample of 12 classrooms stratified by grade as before. Contact will 
be established in the same manner as for the participating classes* 

The sample of teachers will be selected at the same time as the 
sample of classrooms* Each classroom unit selected will have a teacher 
associated with it. The 120 teachers from the 120 classrooms (60 with COP 
aides/60 comparison classes without aides) will constitute the teacher sample. 
Generalizations will be the same as for the classrooms (i.e. , conventional 
F tests will be made using a . 05 level of significance. We may characterize 
the results as if an experiment had been performed. That is, we have used 
an inferential procedure having the probability of a Type I error of . 05). 

The attached letter is a specimen cover letter to Project Directors 
outlining the logistics of the spring survey. 
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Specimen Letter 



Dear (name of Project Director): 

By this time you have undoubtedly spoken to _(name), 

the Abt field staff member responsible for conducting spring testing at ycur 
T,e The entire field staff is now busily preparing for the first two weeks 
in April, when we will complete the COP survey in the sixteen sites. This 
letter will inform you of our plans for the two day visit to your program so 

you can estimate what the logistical requirements Will be for 

(name of site). 

If all goes as planned, the following tasks will be accomplished 
during each of the two days your site is visited. 



Dav One 



1 Post-testing of those COP participants who took the ABLE test 
last fall. The testing will once again take approximately two 
and a half hours and will include both the ABLE test and the 
Participant Questionnaire. The ABLE test is a different form: 
the Questionnaire has been revised. It would be desirable for 
participants to be tested in the same location as before; h-^w- 

we recognize that this will not be possible in all ii.stances. 



ever . 



2. Drop off the self -administer ed questionnaires to: 
The Superintendent of Schools 

Principals of schools in which COP participants are 
assiened to teach 

Administrators /Faculty of the college or university in 
which COP participants are enrolled (to be given to 
those staff members most knowledgeable about the 
COP program and participants in relation to the 
college or university--possibly including instructors 
of COP courses, counselors of COP students, co- 
ordinators of the COP program, etc.) 

The Project Director 

Since most field staff will not know the transportation routes between schools 
or other locations where questionnaires must be dropped off. it J.^ 
most helpful 11 you accompany th.. staff member or, if this is not possible 
provide him with specific directions. The questionnaires take approximately 
ten to fifteen minutes to complete; therefore, it would seem possible to 
collect them toward the end of the business day. 



Day Two 



Post -testing of those COP participants who were unable to be 
tested on previous day. We have reserved this time merely as 
a contincency and hope it will not have to be utilized. 
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2. Pick-up of self -administered questionnaires which were not 
completed the previous day. Again, we hope that this will be 
accomplished at the end of day one. However, if the site is 
very large and includes many respondents, this may not be 
possible. 



Attached to this letter, please find the following lists: 

• Names of those COP participants who were tested in the fall and 
who need to be contacted for post-testing this spring. As you 
know, we cannot substitute new people, even if those on the list 
are unable to attend, as we have no base line data on them. 
Please impress upon the participants the importance of their 
retaking the test once more. 

• Names of school principals, the superintendent of schools and 
college or university faculty whom we wish to full out self- 
administered questionnaires. J£ we have omitted any names or 
if there are any additions or deletions to be made, please 
contact us. 

You are, of course, the best judge in determining the most appro- 
priate time and means of contacting all the people we need to see this spring. 
We look to you to acquaint the personnel listed on the attached sheets with 
the purposes of our visit and in elicting their cooperation with our study with- 
in the next few months. 

[For sites with student/teacher testing, the following will be added.] 

In addition to the above requirement's for the spring survey, plans 
call for student testing at your site. While we have discussed this with you 
and have been in communication with both yourself and your evaluation officer, 
we are setting down the needs for such testing so that the logistical require- 
ments can be addressed. 

We will be administering the Student Opinion Instrument to all the 
sampled classes. We have attached the list of classes and schools where 
we will be testing. A team of three field staff members of Abt Associates 
will be coinine to your site to do the agreed upon testing. In scheduling the 
time for these tests please observed the following restrictions: 

1. Only one field staff member must be assigned to all classes 
within a single grade. That is, if "Miss Smith" of Abt Asso- 
ciates is assigned to one class in grade IV, she must also test 
the other three classes in grade IV. This is necessary to 
keep the administration of the test constant for any one grade. 
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2* For any one grade all testing is to be either in the morning or 
afternoon. That is, all classes in the grade are to be tested at 
similar times of the day. 

We are providing three staff members (two for each grade) in order 
to complete testing at your site in 2-5 days: two days if testing is 
limited to administration of Student Opinion Instrument; five days if testing 
includes achievement tests. 

When you notify the teachers of these classes that we will be coming, 
please tell them that we will have a self -administered questionnaire for them. 
We would like them to respond to the questionnaire during the time we admini- 
ster the Student Opinion Instrument so that we can pick up both at the same 
time. 

We will be calling you in the next few days to discuss our plans more 
fully. Please call us before that if you have any questions or concerns about 
the spring survey. 

Cordially, 



Patricia Cook 
Field Supervisor 
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Brief Description of Plans for Collection, Tabulation and Publication 



The spring survey will be conducted in each .of the 19 sites in April 
of VilZ. On-site interviewing is expected to begin on April 3 and conclude by 
Ap.fil 21. Between two and six man-days v/ill be spent at each site, depending 
on the amount of data to be collected there. On-site interviewing and testing 
will be conducted by members of thp Abt Associates professional staff in 
cooperation with the project staff. The time schedule of the spring survey 
is as follows: 



Janua ry 2 1 

F ebruary- 'March 1 

March 1-15 

April 3-21 
April 24-28 

May 1-19 

May 22-June 16 

July 3-14 
July 14-August 4 
August 14-24 
August 25 



Submission to OE of Request for Clearance 
OMB Approval 

Printing and Production of Instruments As 
Approved by OMB 

Sites Visited and Instruments Fielded 

Protocols Check -in; Field Satisfaction 
(Quality) Check 

Data Processing (Coding, Cleaning, Editing, 
Key Punch) 

Initial Cross -tab Runs and Analysis of Top 
Line Data 

Completion of Tabulations and Analysis 
Completion of Final Report Draft 
Revision of Final Report Draft 
Final Report Submitted 



All data will be processed by Abt Associates Inc. Mr. ixen Carlson of the 
Abt staff will supervise all phases of the data processing. Both machine 
(CDC 6600 Computer) and manual methods will be used. 
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In the following section we describe the questions to be addressed 
by our analysis. In each case the data points required to answer 
the questions are indicated by instrument and item. The letters 
are abbreviations for the instruments; numbers next to them will 
indicate the item number on the instrument. 



Instrument (Questionnaire) Code 

Participant COP 

Teacher T 

''^rir ipal P 

Supe, atendent S 

College Administrator IHE 

Project Director PD 
Adult Basic Learning Examination ABLE 

Student Opinion Instrument SOI 



The three broad hypothesis listed in section A of this supporting 
statement described the general areas within which questions of impact will 
be pursued. More specifically, we will address the following vquestions: 

1. How has the COP experience changed or enhanced the expectations 
of its participants? 

What are their future plans? {^-O'P: 5-6) 

How do they perceive their own performance? (COP: 13, 
21, 23, 24) 

What opinions (positive and negative) do they most 
frequently express? (COP: 1-3) 

What is the relationship between their assignments, 
duties Experiences, and attitudes towards the COP 
program? (COP 7-12, 14 by COP:l-3) 

What demographic characteristics are more frequently 
associated with positive attitudes and professional self- 
image? (COP: Fall Instrument and 15-18 by COP 1, 
13, 24, 25) 

What is the relationship between performance of COP 
participants and their self-image? (ABLE; T: 23, 24, 
20 by COP 13, 21, 23, 24) 

2. What differences in behavior exist between classes which have COP 
aides and classes without aides? 

In types and scheduling of class activities (T:8, 9) 

In number and type of extra -clas s room contacts and 
activities? (T: 10-14; COP; 9-12) 

In amount of individual attention? (T:8, 14f; COP, 7) 
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In performance measures of children? (T: 15; PD, page 4) 

In attitudes of children towards school, self, and others? 
(SOI) 

Are there characteristics other than presence and absence 
of COP aide which is related to differences in performance 
and or attitudes? 

e. g. , size of class (T:4) 
race of teacher, aide (T: 26, 29) 
teacher's attitude towards aide (T: 4-7, 20, 
23-25) 

scheduling of aide's activities (T: 21; COP 8c) 

3, What kinds of chanens, if any, have occurred in selected institutional 

structures concommittant with the onset cf the COP program? 
Elementary school teaching terhinques (P: 1-2) 
Institution of a career lattice (PD: 2; S: 3-7) 
IHE admission, degree requirements (IHE 4-8) 
Community involvement (PD:3,9-11; S: 7) 
Hirinc practires of LEA (S: 1,2) 

Analytic procedures will 'nclude simple summary statistics, 
cross tabs, and, where appropriate, multivariate analysis such as selected 
intercor relation and multiple regression. 

Although some questions are idiosyncratic to the experiences of a 
specific respondent, many of the questions of interest in this study are 
applicable to more than one respondent category. Accordingly, whenever 
analytically appropriate questions were paralleled across categories, it 
gave us data which permits us to either converse on impact data or to in- 
vestigate differences in perception as a function of respondent's position. 
The table below lists all questions paralleled across respondent category. 
The numbers correspond to item numbers within the specified questionnaire. 
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Questions Addressed to More Than a Single Respondent Category: 



Parti- Tea- Prin- Superin- 

Question Areas cipant cher cipal tendent 

Satisfaction with the program 1 20 7 10 

Daily contact with children 7 8 

Time spent in classroom by 

type of activity 8c 9,21 
Extra -school activites 

(school-related) 9-12 10-13 

Achieved goals of COP 19 6 8 

Compairsons with other 

aides /other teacher s 23,24 23,24. 
Changes would like to see 

instituted 5 9 
DiffereAice between COP and 

other aides 25 4 
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D. Documentation of Consultation with Those Supplying Data, Users of 

Data, and Others 

1. The standardized tests to be used were developed by the professional 

staffs of the respective publishers. Data concerning their reliability, validity 
and other psychometric concerns may be obtained from the test manuals. The 
questionnaires and the Student Opinion Instrument were designed by the pro- 
fessional staff of Abt Associates Inc. , primarily by Mr. Kenneth Carlson, 
Ms.. Patricia Cook, Mr. John Doucette and Dr. Sydelle Shapiro. 

In many cases we will use the results of student achievement tests 
administered by the participating LEA's in the course of their regular testing 
program. This constraint becomes necessary in view of the burden an addi- 
tional achievement test would place on the students and teachers involved. 
As most communities conduct standardized achievement testing in the spring, 
we will obtain their results and make the necessary adjustments in the data 
to allow for comparability. In those cases where the LEA's do not conduct 
their own testing, the field staff of Abt Associates will administer the reading 
and arithmetic tests of the Cooperative Primary Test (grades I-III) and the 
Comprehensive Test of Basic Skills (grades IV-VI). These decisions were 
based on extensive discussions with the staff responsible for testing and 
evaluation at the five LEA's. We have been assured that the needed data will 
be available within the time limits dictated by the evaluation work plan. 

The data will be submitted to us in raw score form with conversions 
to grade equivalency or standard scores ^available as needed. Our first 
interest is to compare performance of students by grade and COP aide/no 
aide within each site. Of interest is differences in performance as a function 
of COP aide/no aide within each site. These differences can also be 
used for across site comparisons. Since we do not have pre-test information 
on the students by grade, we are asking the LEA's to send us other information 
on the children which we can use as covariates with the achievement raw 
scores, in this way reducing differences extraneous to the variable of inter- 
est, i. e. , the presence of a COP aide in the classroom. 
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The achievement test for the COP participants will be an alternate 
form (Form B) of the Adult Basic L^^arning Examination (ABLE) Level III 
test, reading and arithmetic computation subtests. This post-testing v/as 
discussed in the previous justification statement, attached herein. 

Re?,p6ndents to the questionnaires in the pretest site, including 
project participants, were interviewed as to the development of the instruments. 
Instruments were also reviewed by the internal technical reviewing staff of 
Abt Associates and by Dr. Robert Hall of OPPE, 

2. Total Burden on Respondents: 

Time per 



Test 




# P sponuents 


Respondent 
(minutes) 


Total Time 
(man-hours) 


Adult Basic Learning 
Examination (ABLE) 


1 


480 


120 


960 


Cooperative Primary 
Test 


3 


360 


90 


540 


Comprehensive Test 
of Basic Skills 


1 


360 


90 


540 


Student Opinion 
Instrument 


1 


3600 


10 


600 


Questionnaire 










Supe r intend ents 


1 


16 


10 


3 


Teachers 


1 


12.0 


20 


40 


COP Participants 


1 


480 


30 


240 


College Administrators 1 


32 


10 


5 


Project Directors 


1 


16 


10 


3 


Principals 


1 


160 


10 

Total Man -Hours 


27 

2958 



The total cost of the entire evaluation effort being conducted by Abt 
Associates Inc. at this time, including the case studies, is $457,000. Of 
this, the estimated cost of the COP survey is $135, 000. 



5 • 0 A3d-0n to Spring Justification Statement 

23 March 1972 

Prior to 0MB clearance, Abt Associates Inc. was requested 
to attempt to provide the Office of Education with specified 
types of inf ori.icition on children. The wire sent on March 23 
to the Office of Education by Abt Associates stated the follow- 
ing : 

"In order to estimate uiie extent to which the experimen- 
tal sample and the comparison sample are similar, Abt Associates 
proposes that the following data be collected from the parti- 
cipating schools: 

1. Current income of family of children in selected 
classes where available from school records. These 
data will be transcribed from school records in 
whatever form they were collected, i.e./ by indivi- 
dual child, by class, or by school level* 

2. Participation in Aid for Dependent Children program 
by children in selected classrooms, where available 
from school records in whatever form they v;ere 
collected (see above) . 

3. Participation in Federal Lunch Program by children in 
selected classrooms where available from school re- 
cords in whatever form they were collected (see above) , 

Those data will be collected from existing school records 
in which they can be foun*-^ and are made available to us and 
will be used to describe the several samples in the present . 
study. 

In addition; we intend to collect the following data which 
were not specified in the original plan: 

4. A list of all Federal, State and Local programs de- 
signed to support the .educational effort over and 
above the normal effort (e.g,, Title I, Title VII, 
etc., programs) which are being administered to 
children in the experimental and comparison schools 
and where available, the selected sample classes 
and children in these schools . 
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5. Achievement scores on those tests \vhich are part of 
our project testing program which were administered 
to sample children at some data prior to our project 
testing activity (i.e., nrof-ect scores on rel a^L 
measures), anri H^hich .u ^u-.^j avan^Lie to'us by" 
the c:oopL;ratin -- ^ chcrjls . 
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b. 0 



5?";;iN'-i SURVEY INSTRUMENTS 



6.1 OTi^r. "inairc"-- 

1 . Participant (COP Aide) 

2 . Teacher 

3 . incipal 

4 . Su per i " ' dent 

^. Coiiege Administrator (IHE) 

6. Project Director 

7. Student Opinionnaire Instrum^irt 

(verbally administered) 

6 . 2 5tandr^r:Ii^ed Tostn 
ABLE , '^tc . 
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2 (1-6) 

2 (7-8) 

2 (9-12; 

2 (13) 



This column is 
for computer 

cod r'ntv - 
C'. T':. • rrar k in 
this ccluTin 



2 V U '.31 



2 (20) 



Trailer Card 
3 (20-21) 



3 (22 231 



0MB Number 51-S72014 

Approval Expires Aug. 31, 1972 




CAREER OPPORTUNITIES PROGRAM: Participant Spring Survey 



This questiannaire is being administered by Abt Associates, Inc. as part 
of a research project for the Office of Education, Washington, D.C. 

We would appreciate your help in our study of the Career Opportunities 
Prcgram, and would like you to fill out this questionnaire. Your answers 
will be used only in this study and will be kept confidential. Please do not 
write vour name on this form; instead, Diease enter the month, date, and 
year of your birth in the blr.' s below. 

Date c; Birth: 



(morth) 



(ri^te) 



(year) 



Please put a check in the bracket that best answers the question. 

When more information is reauired for certain questions, the instructions 
will so indicate. 

Question 1: What best describes your total experience as a COP participant? 

[ ] Ver/ satisfying 

[ i Somewhat satisfying 

i j Neither satisfvinq nor unsatisfying 

i I Somewhat unsatisfying 

j 1 Very unsatisfying 

Question 2: What is it about the COP program you like the most? 



Question 3; What is it about the COP program you like the least? 
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1 

2 
1 
1 
1 
1 
1 
1 
1 
1 

1 

2 



Question 4: What do you think you would be doing this year if the COP 
procram did not exist? 



Question 5: What kind of work do you plan to do next year? (you may 
check more than one J 

[ ] Go out of the teaching field entirely 

[ ] Go into other Federal programs 

[ ] Become a full-time housewife 

[ ] Go into the Armed Services 

[ ] Continue in the COP program 

[ ] Become a full-time student (not in COP) 

[ 1 Leave COP and become a teacher's aide 

[ I Become a full-time teacher 

[ ] Don't know 

[ ] Other (specify) 

Question B: What are your career plans over the next three years? (you may 
check more than one. ) 

( ] Go out of the teaching field entirely 

[ ] Go into other Federal programs 

[ ] Become a full-time housewife 

[ ] Go into the Armed Services 

[ ] Continue in the COP program 

[ i Become a full time student (not in COP) 

f ] Leave COP ynd become a teacher's aide 

( ] Become a full-time teacher 

f ] Don't know 

[ ] Other (specify} 



rnr Cndinq Only 



Question 7: About how many hours in a typical school day do you spend 
with children in each of the following situations? (enter "^o^^ if none j 

Working with one child at a tirr>f? 

Working with small groups of 2 - 5 children 

Working witfi a large part of the class 

Working with the entire class 

Not in direct contact with children 
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Question 8a: Please read the following lists of activities and check all those 
vou have done In the iRst school month. 



nstructional 

] Assisting children in individual instruction 
1 Tutoring more than one child at a time 
] Lesson planning 
] Test construction 
] Giving class lessons 
] Using flash cards 
Administrative 

I Supervising group games 
] Supervising class trips 
] Supervising cafeteria/recess 
1 Supervising study halls 
] /^dnninistering tests 

I Develoni-r. instructional materials (posters, charts, etc.) 

1 Co-~^pitina resource materials 

[ ] Attpodinq ParentTeacher conferences 

] Chp/^H--; .ittpndance/keepinq student records 

j Ordcrir^ ^uDolies 

1 Grading tests 

] Typing 

j Dirtributirr; books 
Technical Assistance 

1 I OoPr.itir.c: P'Oi'^tors 

] Us*ng mimeo. xeroxing equipment 

] OneraTf'^:: othe*' audio-visual equioment 

Question Rb roccrd anv ether class/school activities not listed above. 



Que^t;f>- Qc: rjc*o T-:>t the list in question 8a was divided into four categories. 
Please indrce^re ^^.-.k pprccr.t of your time is spent on each, (it should add ud 

to jrya% i 



!nstnjr:*:?on3l 

A rjnr* in r<:t rat iv/e 

Clerical 



lOCr.o Total 



Q-j&ftio-. ?• -pny fi^ld trios have you gone on with your students in this 

schoci ye^r*^ />np<://v nun-' her) 



Question 10: How many Parent-Teacher conferences have you attended in 
this school year? (specify number) 



Question 11: How many visits to your students' homes have you made this 
school year? (specify number) 



Question 12: How many activities in the school neighborhood have you 
attended this year? (specify number) 



Question 13: Listed below are several problem areas which leacher aides often 
meet up with in class. For each one listed choose the number below that best 
describes how frequently you find it a problem. 

1. Very frequently a problem 

2. Frequently a problem 

3. Occasionally a problem 

4. Seldom a problem 

5. Very seldom a problem 

Number Problem Areas 

Getting children to like me 

fs/laintaining discipline in class 

Getting along with my supervising teacher 

Getting children to work with me 

Giving a class lesson 

Question 14: How many classes have you been assigned to this year? 

[ ] One 

[ ] Two 

[ ] Three or more 

[ ] None 

Question 15: Are you presently living in the community where you are an aide? 

[ 1 No (IF NO, PLEASE SKIP TO QUESTION 17) 
[ 1 Yes 

Question 16: How long have you lived in the community where you are an aide? 

[ ] All my life 

[ ] More than five years 

[ ] One to five years 

[ ] Less than one year 



1 

2 
3 
4 
5 
6 

1 
2 
3 
4 
5 
6 
7 
8 

1 

2 
3 

1 

2 
3 



Question 17: What are your yearly earnings from COP? 

Under Si 000 
$1000- 1999 
$2000 - 3999 
$4000 - 5999 
$6000 ' 7999 
$8000 and over 

Question 18: What is the present total yearly family income; that is, of all the 
members of your household combined? 

Under $1,000 
$1,000- 1,999 
$2,000- 3,999 
$4,000- 5,999 
$6,000. 7,999 
$8,000- 9,999 
$10,000- 11,999 
$12,000 and over 

Question 19: Please tell us how many college/university courses you are 
presently taking. 



Question 20: Did you choos^a the courses you wanted or were thc^y required? 

[ 1 Chose them 

[ ] Took what was required 

[ ] Chose some and took some that were required 

Question 21 : In (general, how would you rate the courses offered by the college? 

f 1 Too difficult for me 
[ 1 About right for me 
( 1 Too easy for me 

Question 22: About how many hours a week do you spend on your college 
work m thp follnwmg situations? 

Attending college classes liours 

COP work experience hours 

Doing homework or reading 

for college courses hours 



66 



Question 23: Comoared with most other aides, how would you rate yourself 
on the following? 



ERIC 



3 (371 
3 (38) 
3 (39) 
3 (40) 
3 (41) 

3 (42) 

3 (43) 
3 (44) 

3 (4S? 



2 (73.74) 
7 (7B.7fi) 



Better than About the Not as good 







other 
Aidr.'S 


same as 
others 


as other 
Aides 


3 (26) 


Knowledge of subject matter 


[ ] 1 


[ ] 2 


[ ] 3 


3 (27) 


Presenting lessons to class 


[ ]i 


I ]2 


[ 13 


3 (28) 


Gettino iHpa<i armcc 


f 1 1 


I J 2 


I 1 3 


3 (29) 


VJCLLIMLj cJIUMLj Wllfl pdrcnib 


I J 1 


[ ] 2 


[ ! 3 


3 (30) 


Understan the needs of 
children of inis neighborhood 


[ j 1 


[ ]2 


[ 13 


3 (31) 


Getting along with children 


[ n 


[ ]2 


[ 13 


3 (32) 


Ability to instruct children 
individually 


[ ]i 


[ ]2 


I )3 


3 (33) 


Ability to tutor 


[ ]i 


[. ]2 


[ ]3 


3(34) 


Ability to instruct children 
in small grouos 


[ ]^ 


i ]2 


[ ]3 


3 (35) 


Lesson planning 


I ]i 


[ ]2 


[ 13 


3 (36) 


College performance 


[ ] 1 


[ j2 


[ ]3 



Question 24: Compared with your supervising teacher(s), how would you 
rate yourself on the following? 



Better than 
your teacher 



Knowledge of subject matter 

Presenting lessons to class 

Getting ideas across 

Getting along with parents 

Understanding the needs of 
children of this neighborhood 

Ability to instruct children 
individually 

Ability to tutor 

Ability to instruct children 
in small grouos 

Lcsnn'". planning 



Same as 
your teacher 



J2 
]2 
]2 
]2 



Not as good 
as your teacher 



13 
]3. 
)3 



)3 

)3 
)3 

]3 
]3 



Question 25: Do you have any additional comments? If you do, please write 
them here. 



Thank you for your cooperation and assistance! 



PS 



For Codinq Only 
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questionnaire: for teachers 



This questionnaire is being administered by Abt Associates Inc. as 
part of a research project for the Office of Education, Washington, 
D. C, 

Please help us i 

We would like to ask you a few questions to aid us in our study of the 
Career Opportunities Progrann. Your answers v/ill be used only for 
the purposes of this study- Do not write your name on this form! 
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ixi tl 

1 
2 
3 

1 
2 
3 

1 
2 
3 

1 
2 
3 

1 



OMB Number 51-S 72 014 
Approval Expires Aug-31, 1972 



Do not write in this box; it is for coding only 
IMPKD 2 
W 

SITE 



Please put a check [ V^] in the bracket that best answers the ques- 
tion- When rr>ore information is required for certain questions, 
the instructiont: will so indicate* 

Question 1. How long hav^ you been a full-time teacher? 

[ ] Less than one year 

[ ] 1-3 year s 

[ ] 3 or more years 

Question 2. How long have you taught in your present school? 

[ ] Less than one ysar 

[ ] 1-3 years 

[ ] 3 or more years 

Question 3, Please circle the grades you. are presently teaching. 
1 2 3 4 5 6 

Question 4. How many students are there in your class(es)? 
students 

Question 5. Given a choice, would you want a teacher aide in 
ycur room? 

[ ] No 

[ ] Yes 

[ ] Don't Know 

Question 6. Did you have a choice about having a teacher aide 
in your room this year? 

[ ] No 

[ ] Yes 

[ ] Don't Know 

Question 7. Have you had a teacher aide in the past? 

[ ] No 
[ ] Yof 

Question fi. About how many hours in a typical school day do you 
spend with children in each of the following situations? (ENTER 
"0" IF NONE. ) 

Working with one child at a time 

Working with small groupg of 2-5 children 

' Working with a large part of the class 

Workinf^ with the entire class 

Not in direct contact with children 
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Card 2 



(21) 
(22) 
(23) 
(24) 
(25) 

(26) 



(27) 



(28. 29) 
(30, 31) 
(32,33) 
(34, 35) 



(36, 37) 
(38, 39) 
(40.41) 
(42, 43) 



EKLC 



Question 18. Five goals of COP arc listed below. Please tell us 
how well you think these poals are being met in your school. 
Rank thenn on a scale from I fmost successful) to 5 'least suc- 
cessful). (ENTER A '1* TO INDICATE WHICH OF THESE GOALS 
HAS BEZN MOST SUCCESSFULLY MET; ENTER A '2» TO INDI- 
CATE T/HE GOAL NEXT MOST SUCCESSFULLY MET AND CON- 
TINUE TO ^5^ TO INDICATE THE GOAL LEAST SUCCESSFULLY 
MET. ) 

To attract low-income people to a career in education 

To encourage parent and community participation 

To improve teacher training through a work-study program 

To raise achievement levels among low-income children 

To improve the efficiency of professional staff time 



Question 19- What part of the time have you had an aide in your 
class this year? 

[ ] Full-time 

[ ] At least 50% of the time but less than full-time 
[ ] Less than 5 0% of the time 

Question 20. Please check the category that best describes your 
-total experience in uJing an aide. 

[ ] Very satisfying 

[ ] Somewhat satisfying 

[ ] Neither satisfying nor unsatisfying 

[ ] Somewhat unsatisfying 

[ . ] Vf»ry unsatisfying 

Question 2 1. What percentage of your aide' s time is spent on 
each of the following? (SHOULD ADD UP TO 100%. ) 

% Instructional (e.g., lesson planning, class lessons, 

constructing tests) 

Administrative (e.g., supervising class trips, recess, 

giv.int^ teats, developing instructional materials, etc) 

Clerical (e.g. , ordering supplies, keeping records, 

typing, grading papers) 

Technical (e.g.. operating audio-visual equipment, 

running copyin.g machines) 

100 % TOTAL 

Question 22. B efor o you had an aide, what percentage of your time 
was spent on eacn oi the following? (SHOULD ADD UP TO 100%. ) 

^^^o Instructional 

Administrative 

Clf^rical 

Technical 

ion TOTAL 



ERIC 



(33, 34) 
(35, 36) 
(37, 38) 
(39,40) 

(4W42) 

(43,44) 

(45,46) 

(47,48) 



(49) 
(50) 
(51) 
(52) 
(53) 

(54) 



(55, 56) 
(57, 58) 

(59) 



(60) 



1 
2 
3 



1 
2 
3 



Question 9, What percentage of your time is spent on each of the 
following categories? (SHOULD ADD UP TO 100%, ) 

% Instructional (e.g., lesson planning, class lessons, 

constructing tests) 

% Adnninistrative (e.g., supervising class trips, recess, 

giving tests, developing instructional material) 

Clerical (e.g., ordering supplies, keeping records, 

typing, grading papers) 

Technical (e. g, , operating audio-visual equipment, 

running copying machines) 

100 % TOTAL 

Question 10. How many field trips have you gone on with your 
students in this school year? (SPECIFY NUMBER) 

_^ trips 

Question 11. How many parent-teacher conferences have you 
attended in this school year'' ^SPECIFY NUMBER) 

conferences 



Question 12, How many visitF to your students' homes have you 
made in this school year? (SPECIFY NUMBER) 



VIS its 



Question 13. How many activities and events in the school neigh- 
bornood have you attended in this schoofyear? (SPECIFY NUMBER) 

activities and events 

What were they? (PLEASE SPECIFY) 



Same More 



Question 14. How does you. .ctivity in this school year compare 
to last year with regard to the following areas? 

Less 

a. Contact with parents 

b. Attendance at workshops 

c. Enrollment in college courses 
Out-of-class activities for your students i 

Use of innovative techniques in your 
classroom 

Time spent in diagnosis of individual [ 
learning difficulties in your classroom {1) 



d. 
e. 

f. 



r 



(2 



[ 



(3 



^?>'l^^J'ifV^• Please estimate the following for your class for 
tne last two we eks: 

number of times tardy 

. number of absenceii 



?n^''^^i°.'J. ^'"^ ^^^^ ^^^^ ^ at least 

30/0 of the time last year? 

[ ] No aide 

[ ] Yea, COP aide 

[ ] Yes, other aide 

Question 17. Please check the kind of aide assistance you now 

[ ] COP aide (CONTINUE ON TO QUESTION 18) 
[ ] Other aide (SKIP TO QUESTION 19) 
[ ] No aide (SKIP TO QUESTION 27) 
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(44) 
(45) 
(46) 
(47) 

(4 8) 
(4 9) 

(50) 
(51) 

(52) 



(53) 
(54) 
(55) 
(56) 
(57) 

(58) 
(59) 

(60) 
(61) 

(62) 
(63) 



Question 23. Compared with other aides, how do yon rate your 
prosont nirif on the fnllowin??? (CHECK TflE COLUMN THAT 

^"'^'"^^ ^"^^ yOVR AIDERS PERFORMANCE 

AS COMPARED TO THAT OF OTHER AIDES. ) ^xn^j:. 



Better than 
other Aides 



Knowledge of subject matter 

Presenting lessons to class 

Getting ideas across 

Understanding the needs of 
neighborhood children 

Getting along with children 

Ability to instruct children 
individually 

Ability to tutor 

Ability to instruct children 
in small groups 

Lesson planning 



About 
the same 



Not as 
good 



(1 



(2 



(3 



Question 24. How does your aide's performance compare with 
that of an average teacher on the following? 

Bette r than 

the average About Not as 

teacher the same good 



Knowledge of subject matter 

Presenting lessons to class 

Getting ideas across 

Getting along with parents 

Unde rstajiding the needs of 
neighborhood children 

Getting along with children 

Ability to instruct children 
individually 

Ability to tutor 

Ability to instruct children 
in small groups 

Lesson planning 



(1) 



(3) 



Uuesticii 25. Do y^^ =ee differences between COP aides and 
other teacher aides? 

1 [ ] No 

2 [ ] Yps 

3 [ ] Don't know 
IF YES, what are the differences? (PLEASE SPECIFY) 



(64) 



ERIC 



Question 26. Please indicate which ethnic / cultural group your 
aide b elongs to, 

A [ ] American Indian 

2 [ ] Black 

3 [ ] Oriental 



4 
5 

7 
8 

1 

2 

1 
2 
3 
4 

5 

1 

2 

3 

4 

5 
6 

7 



Spanish: 

J Puerto Rican 

] Othrv 

] White other than Spanish 

J Other (SPECIFY) 



Question 27. Ploase indicate your sex, 
[ ] Male 
r ] Female 

Question 28, Which age group are you in? 

[ ) 20-24 

[ ] 25-29 

[ ] 30-34 

[ ] 35-39 

[ ] 40 and over 

?eTong°o.''' ^^^''^^ ethnic /cultaral group ^ 

[ - ] American Indian 

[ ] Black 

[ ] Oriental 

Spanish: 

[ ] Mexican 

[ ] Puerto Rican 

[ ] Other 

[ ] White other than Spanish 

[ ] Other (SPECIFY) 



Question 30. Please use this space for additional corrLments: 



Thank you for your cooperation and assistance! 
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QUESTIONNAIRE FOR PRINCIPALS 



This questionnaire is being administered by Abt As sociates Inc. as 
part of a research project for the Office of Education. Washington, 



D. C 
Please help us 



We would like to ask you a few questions to aid us in our study of the 
Career Opportunities Program. Your answers will be used only for 
the purposes of this study. Do ^^ot write your name on this form! 



Er|c 7 4 



ERIC 



( i- 6) 
( 1- 8) 
( 9-12) 
(13) 

This column 
is for computer 
coding only- -do 
not mark in this 
column 

(20) 




(21) 

(22) 
(23) 
(24) 
(25) 

(26) 

(27) 
(28) 



(29) 
(30) 
(31) 
(32) 
(33) 
(34) 

(35, 36, 37) 
(38) 



(39) 



1 

2 



1 

2 
3 



OMB Number S j^^z 0 14 
Approval ELcpires~7CugT3l, 1972 



Please put a check [✓Tin the bracket that best answers the ques- 
tion. When more information is required for certain questions, 
the instructions will so indicate. 

Question 1. Do you feel there have been any changes in the acti- 
vities of your classroom teachers since the COP program was 
instituted in your school? 



] No 
] Yes 

[ ] 

[ ] 

[ ] 

[ ] 

[ ] 



g CHECK ALL OF THE CHANGES BELOW THAT 

HAVE OCCURRED: ^h-x^^vy 1 iriA 1 

Fewer non-instructional (clerical, administrative) 
tasks to handle 

More time to work with individual children 
More time to work with the entire class 
More contact with parents 

increased attendance at lectures, in-service workshops 
conferences, etc. *^ ' 



[ ] 



] Other (SPECIFY) 
Other (SPECIFY) 



Question 2. Which of the following imoacts do vou feel the COP 
program has had on the structure of your school? (CHECK AI I 
APPROPRIATE ITEMS. ) 

[ ] None 

[ ] Increased scheduling flexibility 

] Introduction of a Career Lattice 

[ ] Transition to an "open classroom" model 

L ] Increased tutoring of individual children 

[ ] Team Teaching 

[ ] Other (SPECIFY) 

Question 3. Were there aides in your school before COP? 
[ ] No ( IF NO, PLEASE SKIP TO QUESTION 5. ) 
C ] Yes 

?ercheraides?° ciifferences between COP aides and other 

t ] No 

[ ] Yes 

[ ] Don't Know 

YES, what are the differences? (PLEASE SPECIFY) 
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(40) 

(41) 

(42) 

(43) 

(44) 

(45) 
(46) 
(47) 



(48) 
(49) 
(50) 
(51) 
(52) 



Question 5. V/>iich of the followinR changes would you like to see 
instituted refiardinp COP aides in vour school? (CHECK ALL 
APPROPRIATE ITEMS. ) 

) More COP aides in legular classrooms 

] Fewer COP aides in regular classrooms 

] More supervision of COP aides by training staff 

] More COP aides to work with '^special'* children 

3 More COP aides to work with minority group children 

] Other (SPECIFY) 



Question 6. Five goals of COP are listed below. Please tell us 
how well you think these goals are being met in your school sys- 
tem. Rank them on a scale from 1 (most successful) to 5 (least 
successful. (ENTER A TO INDICATE WHICH OF THESE 
GOALS HAS BEEN MOST SUCCESSFULLY MET; ENTER A '2' 
TO INDICATE THE GOAL NEXT MOST SUCCESSFULLY MET 
AND CONTINUE TO '5' TO INDICATE THE GOAL LEAST SUC- 
CESSFULLY MET. ) 

To attract low-income people to a career in education 

To' encourage parent and community participation 

To improve teacher training through a work-study program 

To raise achievement levels among low-income children 

To improve the efficiency of professional staff 



(53) 



I 

2 
3 
4 
5 



Question 7. Please check the category that best describes your 
otal experience in having COP aides in your school. 

] V^^ry satisfying 

] Somewhat satisfying 

] Neither satisfying nor unsatisfying 

] Somewhat unsatisfying 

] Very unsatisfying 



(54) 



• Question 8. Please use this space for any additional comments. 



EKLC 



Thank you for y-oui cooperation and assistance? 
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QUESTIONNAIRE FOR SUPERINTENDENTS 



This questionnaire is being administered by Abt A s sociates Inc. as 
part of a research project for the Office of Education, Washington, 
D. C. 

Please help us! 

We would like to ask you a few questions to aid us in our study of the 
Career Opportunities Program. Your answers will be used only for 
the purposes of this study. Do not write your name on this form! 



ERIC 



OMB Number 51-5 72 014 
Approval Expires Aug. 1972 



( 1- 6) 
( 8) 
( 9-12) 
(13) 

This column 
is for computer 
coding only- -do 
not mark in this 
column 



(20) 
(21) 
(22) 
(23) 

(24) 

(25,26,27) 



(28) 



(29) 

(30, 31, 32) 
(33) 



1 
2 
3 



(34-36) 
(37-39) 

(40-42) 



(43-45) 



(46-48) 



Do not write in this box; it is for coding only 
IMPED 2 
W 

SITE . 



Please put a check [yXf in the bracket that best answers the ques- 
tion. When mo^*- information is required for certain questions, 
the instruction will so indicate. 

Question 1. Which of the following changes in hiring procedures 
have occurred in your school system over the past two years? 



.No longer 
required/ 
allowed 



Now 

required/ 



al 



a. National Teachers Exa.m [ 

b. Full State Certification [ 

c. Personal Interviews [ 

d. Hiring of Aidf-.'7/Teacher s who have [ 
not met state certification requirements 

e- Partial/Temporary Certification [ 

f. Other (SPECIFY)_ [ 



[ ] THERE HAVE BEEN NO CHANGES (Skip to Question 3. 

Question 2 Do you attribute any of the above changes to the 
presence of COP aides in your school system? 
[ ] No 



owed 



) 



[ ] Yes IF YES, which changes- 



Question 3, Is there a Career l^attice operating within your 
school system? ' 

' [ ] No 

[ ] Yes 

[ ] Don't Know 

Question 4. How many teacher aides are in your system? 

teacher aides 

How many of these are COP aides? 
■ COP aides 



Question 5, How many of your present aides were aides in this 
school system last year? 

^aides 

Question 6. What is the next highest position after "teacher aide* 
in your school system? 



How many aides have moved to this position (or higher) in the 
last year? 



aides 



How many of these aides were COP aides? 
COP aides 
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(49) 

(50) 
(51) 

(52-54) 



Question 7. Since the inception of COP within your school system^ 
have any of the following changes occurred? 

[ ] Increased linkages between the school syatenn and community 
groups (Model Cities, etc. ) 

[ ] Increased access to Federal funds 

[ ] Increased discussions between your system and the local . 
Teachers' Union 

[ ] Other changes not listed above (SPECIFY) 



(bS) 
(56) 
(57) 
(58) 
(59) 



(60) 
(61) 
(62) 
(63) 
(64) 
(65) 



(66) 



1 
Z 
3 
4 

5 



(67) 



Question 8. Five goals of COP are listed below. Please tell us 
how well you think these goals are being met in your school sys- 
tem. Rank them on a scale from 1 (most successful) to 5 <least 
succesVfirr). (ENTER A ' 1 ' TO INDICATE WHICH OF THESE 
GOALS HAS BEEN MOST SUCCESSFULLY MET; ENTER A 
TO INDICATE THE GOAL NEXT MOST SUCCESSFULLY MET 
AND CONTINUE TO '5' TO INDICATE THE GOAL LEAST SUC- 
CESSFULLY MET. ) 

To attract low -income people to a career in education 

To encourage parent and community participation 

To Improve teacher training through a work-study program 

To raise achievement levels among low -income children 

To improve the efficiency of professional staff 



Question 9. Which of the following changes would you like to see 
instituted, regarding COP aides in your school system? (CHECK 
ALL APPROPRIATE ITEMS. ) 

] More COP aides in regular classrooms 

] Fewer COP aides in regular classrooms 

] More supervision of COP aides by training staff 

] More aides to work with "special" children 

] More COP aides to work with minority group children 

] Other (SPECIFY) 



Question 10. Please check the category that best describes your 
total experience in liaving COP aides in your school system. 

] Very satisfying 

] Somewhat satisfying 

] Neither satisfying nor unsatisfying 

] Somewhat unsatisfying 

] Very unsatisfying 

Question 11. Please use this space for any additional comments. 



Thank -you for your cooperation and assistance! 
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QUESTIONNAIRE FOR COLLEGE ADMINISTRATORS 



This questionnaire is being administered by Abt As sociates Inc. as 
part of a research project for the Office of Education, Washington, 
D.C. 

Please help us ! 

Wii would like to ask you a few questions to aid us in our study of the 
Career Opportunities Program. Your answers will be used only for 
the purposes of tlais study. Do not write your name on this form! 



OMB Number 51-S72 014 
Approval Expires Aug. 31, 1972 



( 1- 6) 
( 7- 8) 
( 9-12) 
(13) 

This column 
is for computer 
coding only--do 
not mark in this 
colunnn 



(20) 
(21) 
{21) 
(23) 
(24) 

(25) 



(26) 



(27) 



(28) 

(29) 
C30) 
(31) 

(32) 



1 
2 
3 
4 

5 
6 



Do not ^v^ite in this box; it is for coding only 
IMPED 1 



W 



SITE 



Please put a check [V^j in the bracket that beet answers the ques- 
tion. When more ixid^ormation is required for certain questions, 
the instructions v,iH so inaicate. 

Question l{a) Over the past two years, do you feel that the 
Career OpporLunitie? Program (COP) has ?ji any way affected 
any of the following? (FOR ALL THOSE YOU 'CHECK, PLEASE 
INDICATE HOW OR I1^I WHAT WAY, AND THE COURSE INVOLVED. ) 



HOW/WHAT COURSE? 



[ ] Course content 

[ ] Course additions 

[ ] Course deletions 

[ ] Course format 

[ ] Course meeting times 



Question 1(b) Are these coursa:i open to non-COP students? 
[ ] No 

[ ] Yes IF YES, which courses? 



Question 1(c) Who initially suggested the changes described 
in Question 1(a) 

[ ] Facult/ 

[ ] Administration 

i COP stu ents 

[ J COP staff 

[ ] Local sch»:-ols 

[ ] Other (SPECIFY) . 



Question 2. Have you hired any faculty members especially 
for the COP program? 

[ ] No 

[ ] Yes IF YES , are any of them: 

I ] Specialists in training pa raprof es sional t" to become 
teachers? 

[ ] Personnel from the local school system? 
[ ] Representatives of minority groups? 
[ ] From low-income groups? 

Question 3, Have there been changes in admissions requirements 
over the past two years in the Department (or school) of Education? 

[ ] No (IF NO, SKIP TO QUESTION 6) 

[ ] Yes 



(33) 
(34) 
(35) 
(36) 

(37,38,39) 
(40) 



(41) 



(42) 



Question 4. Which of the following changes in admissions re- 
quirements have been instituted? 

[ ] Acceptance of a G^D diploma 

[ ] Lowering of SAT score cut-offs 

[ ] Lowering of Achievement Test score cut-offs 

[ ] Change in weight assigned to College Board Scores 

[ ] Other (SPECIFY) 



Question 5. Do these changes in admissions requirements affect 
all students or do they only apply to specific groups? 

[ ] AU students 

[ ] COP students only 

[ ] Other groups (SPECIFY) 

Question 6, Have there been any changes in degree requirements 
over the past two yes.rs? 

I 1 No (IF NO, SKIP TO QUESTION 8) 

[ ] Yes (IF YES, PLEASE SPECIFY WHAT THESE CHANGES 
HAVE BEEN) 



Question 7. Do these changes in degree requirements affect 
all students or do they only apply to specific groups? 

[ ] All students 

[ ] COP students only 

[ ' Other groups (SPECIFY) 



-(43) 



Question 8. Please use this space for any additional comments. 



Thank you far your cooperation and assistance! 



ERIC 



QUESTIONNAIRE FOR PROJECT DIRECTORS 



This questionnaire is being administered by Abt As sociates Inc. as 
part of a resea-ch project for the Office of Education, Washington, 



D. C 

Please help us ! 



We would like to ask you a few questions to aic" us in our study of the 
Career Opportunities Program. Your ans^vers will be used only for 
the, purposes of this study. Do not write your name on this form! 



OMB Number 5i-S72 014 
Approval Expires Aug, 31, 1972 



EKLC 



( 1- 6) 
{ 7- 8) 
( 9-12) 
(13) 

This column 
is for compute V 
coding only- -do 
not mark in this 
column 

(20-22) 



(23) 



1 

2 
3 



(24) 



1 
2 



(25) 
(26) 
'27) 
(28) 
(29) 
(30] 
(31) 

(32) 



(33) 
(34) 
(35) 



(36) 
(37) 
(38) 



Do not writ 
IMPED 2 



box; it is for coding only 



SITE 



Please put a check [ ] L the bracket that best answers thu ques 
tion, 'vVlien more information is required for certain questions, 
the instructions will ao indicate. 

Question 1, What is the total number of COP aides presently 
active in your program? 

COP aides 

Question 2, Do you employ a Career Lattice for COP aide • 
advancement? 

[• ] No 

[ ] Yes 

[ ] Don't Know 

^ YES, please indicate each level, the approximate number of 
aides on each level and the pay scale for each liivel. 

J-EVEL # AIDES PAY SCALE 



C ] N 
[ ] ^ 



("F'or coding only) 



Question 3. Please ir^dicate if any of the following outside com- 
munity agencies have expressed an L^.terest in obtaining th'* ser- 
vices of COP aides. 

[ ] Day care centers 

] Private schools 

] Parochial schools 

] Anti-poverty agencies 

] Other (SPECIFY)^ 



Question 4. Have th-;re been requests for additional COP aides 



in the public s- lie 
[ 1 No 
L ] Yes 



/ou are servicing 



: ;tion 5. Thr^v.^^h «/hat source(s) do you fund your COP program? 



Question 6. What are your present recruil-ment procedures 
for COP pa. t: - .mts? 



(3V) 



(40) 
(41) 
(42) 



(43) 
(44) 
(45) 
(46) 



(47^ 
(48) 
(49) 

(50) 



Question 7. Were your recruitment procedures for this past 
year the same as or different from the first year of the program? 

[ ] Same as first year 

[ ] Different from first year (PLEASE SPECIFY) 



Question 8. Who determines which college/university courses 
a COP participant may take? 



Question 9. Do you have a COP Advisory Board/ Council? 

[ ] No (IF NO, SKIP TO QUESTION 13) . 

[ ] Yes IF YES, who is represented on the Advisorv Council'^ 
(LIST BY TITLES AND/OR OCCUPATION.) 



Question 10. How often does this council meet? 



(51) 
(52) 
(53) 
(54) 

(55) 
f56) 

(57) 



Question 11. What is the Council's role in the following areas? 
(CHECK APPROPRIATE BOXES.) 

No 



Sets 



role Advisory Guide 



Recruitment of participants 

Admission of participants 

Course requirements 

Participant assignments 
in practicum 

Project staffing 

Budget 



mes 



Makes 
Operating 
Decisions 



Question 12. Are you a member of the Board or Council? 
1 [ ] No 

^ [ ] Regular membe* 

3 [ ] Ex-officic member 



(58) 



Question 13. Please use this space for any additional comments. 




1972 



5 4 3 2 1 




5 . 4 . . 3 2 . 1 




5 4 3 2] 




III. FIEL DING THE STUDY 
INTRODUCTION 

In the Operational Plan of 29 July 1971 the stated goals 
were the measurement of performance and career aspirations of COP 
aides; the comparison of performance and selected behaviors 
of students who have COP aides with those who have not; and a 
description of institutional changes that have taken place during 
the two years of OOP's exist-nce. Our survey efforts were accord- 
ingly geared towards the testing of the following three hypotheses: 

o COP has been an effective tool in improving the edu- 
cational experi. ace of students in the classroom; 

• COP has bee fective in enhancing the career po- 
tentials of its participants; 

# COP has engendered change in institutional structures. 

In order to test these hypotheses, data was obtained from 
three primary sources: participants, institutions and students. 
Although the impact on students is OOP's primary concern, the 
effect of the program on the aides themselves is of nearly equal 
importance. If the program is unsuccessful in training aides, 
no impact on students can be expected. Similarly, even if no 
impact is detected on student achievement, the establishment of 
an enhanced set of reality-based aspirations on the part of the 
participants would constitute a degree of success. An expressed 
goal of COP (Project Directors Handbook, COP Leadership Training 
Institute, 1970, p.l) is 

..."to attract capab]<^ persons to careers in education in 
a ay that will improve both education and employment 
opportunities for the poor, and establish career lattices 
in schools so that productive careers can be followed by 
those recruited through this program." 
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To explore this, partj-cipants were administered & standardized 
test of basic skills and a questionnaire designed to elicit 
information concerning career plans and aspirations; attitudes 
towards the program, students, peer? and teachers. 

In order to make longitudinal comparisons ofi the parti- 
cipants, we investigated their changes on these dimensions over 
time. This was accomplished by administering the standardized 
test and the r^ues tionnaire to the same participants on both a 
pre-test and post-test basis. All data was collected during two 
major field efforts, one in the Fall of 1971, the other in the 
Spring of 1972. 

The first two chapters of Volume III present the Opera- 
tional Plan of July 29, 1971 and the developmental history of 
both Spring and Fall field instruments along with their re- 
spective justification statements. This chapter will describe 
in detail the field efforts for both Fall and Spring testing, 
including field preparation (of both COP Directors and Abt 
staff), field procedures and results, and the survey of the State 
Education Agency (SEA) certification officers. 
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1 . 0 Fall Survey Proce '':ires 

Abt Associates Inc * recognizes that good field procedures 
are an integral part of conducting effective research in the 
social sciences. V7ith regard to this study of the Career Oppor- 
tunities Program, we realized that the Fall field effort would 
in large part decermine our credibility with the participants 
and Directors of the sixteen (15) sites, which wCyUld in turn 
affect our access tc sites in the Spring and our ability to collect 
unbiased data. The following sections describe in detail the 
field effort for Fall testing, including- field preparation 
(both COP Directors and Abt staff), field procedures and results, 
and the State Education Agencies survey. 

1 * 1 Conference with COP Directors 

Originally^ sixteen COP sites were chosen from a sample 
frame defined as COP projects with no fewer than 3 0 and no more 
than 200 participants. The sample was drawn by probability pro- 
portional to size (PPS) . 

The Project Directors of each of these sixteen sites were 
invited) to a conforence on 24 September 1971 in Washington, D.C, 
where the operational plan was fully explained. Prior to the 
meeting, they received invitational letters from Wilton Ander- 
son, former Chief of the Career Opportunities Program, and Abt 
Associat c, who made all necessary arrangements and paid all 

expenses. In addition, the sixteen Project Directors were sent 
a copy of the operational plan prior to the meeting. • 

Representatives from all sixteen sites attended the meet- 
ing. The Directors, while concerned about the testing proce- 
dures (esp6 jially in those sites that called for student testing), 
were both interested and cooperative. Documents related to this 
conference may be found in Section IV. 
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1,2 Field Staff Preparation 

The eight Abt Associates field staff chosen to participate 
in the COP Fall testing were all experienced in conducting field 
work. Therefore r the bulk of field preparation consisted of 
familiarization with the COP program and with their respective 
testing materials . 

Two meetings for the entire field staff were held before 
on-site visits were made. The first meeting, held after assign- 
ment of field staff, accomplished the following: 

e Staff /site matching; 

• Familiarization with the COP program — overall purposes, 
goals and evaluation needs; 

• Distribution of COP Fall Survey Field Guide; and 

• Clarification of field procedures — schedules, contacts, etc 

The second full staff meeting met shortly before the field effort 
and covered the following: 

o Procedures f or adiriinistering the ABLE tests and parti- 
cipant questionnaires; 

o Final testing arrangements made with COP Directors; 

• Distribution of materials; and 

• Special attention to briefi:ag of participants before 
testing began . 

The field guide written for the staff is contained in Section 
III. The on-site schedule appears below: 

Saturday 

10/16 San Diego, California 

Monday 

10/18^ Lafayette Georgia; Cedar Rapids, Iowa 

Tuesday . . 

10/19 Las Cruces, New Mexico; New Orleans, Louisiana; 

Charleston, West Virginia; Belcourt, North Dakota 

Wednesday 

10/20 Lakewood, Colorado; Jackson, Mississippi; Hartford, 

Connecticut; Lillington, North Carolina; Belcourt North 
Dakota 



Thursday 

10/21 La]c£wood, Colorado; Jacksonville, Florida? Spokane, 

Washa,ngton. ^ 

Friday 

10/22 Asheville, North Carolina; Stockton, California; 

Los Angeles, California 

No serious problems were encountered at any site. The 
COP Directors were very helpful, although some were concerned 
about the possible reaction of their aides to the testing. 
Since most of the Directors had discussed the evaluation with 
the aides before Abt staff arrivals, the testing went smoothly 
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1.3 Schedule for Fall Field Effort 



Date 



Event 



Week of 
September 13 



Letter sent from Washington , D • C . to Pro j act 
Directors telling them of meeting on September 24 
(This letter had an attached copy of our work 
statement • ) 



September 17 



Letter enclosing tickets, etc, sent ro Project 
Directors from Abt. In it we asked for a list 
of COP aides with all first-year aides to be 
designated by an asterisk. 



September 22 



Each field staff member called hi;Z Pro j ect Dir-- 
ector to establish contact and fiiid out if he re- 
ceived the ticket, if he developed tJ.e aides 
list and would bring it to Washingtot! . 



September 2 4 



Washington, D.C, meeting with COP Project Direc- 
tors. At this meeting we got names of the person 
who w^.s to act as a liaison at each site, 



Septeriiber 3 0 



October 5 



The liaison at each site was called and contact 
established- The liaison was informed of the 
following procedural steps to be accomplished: 

9 We would send him a list of the aides who 
formed the sample for his group. 

o We would arrange an agreed upon date or veri- 
fy one chosen at the Washington, D.C. meeting. 

• We would discuss survey logistics: room, 
place, time for tests. The duration was to be 
3 hours and testing at the IHE was highly 
preferable. 

• A local person, preferably the liaison, was 
to be present at the testing. 

• The Abt staff member would call again on 
October 5. 

The liaison was called and verification was made 
regarding receipt of the participant list and test- 
ing arrangements. Arrangements were also made for 
a meeting place and time of meeting. 
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1 . 4 Field ProceQur as and Results 

Every attempt was made to test participants during regular 
COP time — either in the early or late afternoon r vrtxen they would 
usually be attending college classes. However^ the San Diego 
site was unable to arrange a time for participants to be tested 
other than a Saturday morning . Therefore^ Abt Associates agreed 
to pay San Diegc participants to compensate for their time. 

A sign-up sheet, which appears in the Fall Field Guide, -wa 
was used to assure correct participant se].ection for Spring re- 
testing. We were able to keep a tally of the total nxamber of 
participants who appeared for testing at each site; and to begin 
to tackle the question of motivation during Spring testing. The 
average number of sampled participants who did not show up for 
testing across the sites was three. San Diego had the largest 
number of sampled non-respondents (110, whereas in six sites all 
sampled par t i.cipants . s ; wed up fCL ^ j. 

. ai t's ^. 'iv-rn a shcrt , .^s ior : ::ai : pri- 
'..arily f. :z.sd on -em; ^r-^". ic infcrmd. "icn. "n a. xt_"^-:. demo- 
graphic in: -nation the c .^s were aske:: descr ..3 their career 
plans and to explain how they first h'eard of COP. They also 
listed the classes to which they were currently assigned on 
the sign-in sheet. Finally, the reading and math (problem- 
solving) sections of the Adult Basic Learning Examination (ABLE) 
were administered. (Adies were retested on ABLE in the spring 
survey ana were asked to respond to a longer questionnaire*) 

The Abt field staff reported a high degree of cooperation 
by both COP Directoris and participants. Ail staff felt that 
testing went smoothly and that a good working rapport had been 
established within the sites. This perception v/as validated by 
satisfaction checks made by telephone by the Field Coordinator, 
All the Project Directors contacted felt that the effort proceeded 
smoothly . 
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1 . 5 Sea Telephone Survey 

One implicit goal of COP is to effect changes in state 
teacher certification regulations to better accommodate the ca- 
reer experiences of teaching paraprof essionals • In order to 
assess OOP's success in meeting this goal Abt Associates Inc. 
conducted a telephone survey of state certification officers 
in SEA to determine the extent to which changes have occurred or 
are planned in each state and the extent to which the certifi- 
cation officers was their changes as responding to the needs of 
COP participants. 

The survey began with a letter of introduction^ giving 
a brief description of the study (but not mentioning COP) and 
Abt Associates Inc , sent 10 Certification Officials in the fifty 
rotates, as well as in the Distric : or Columbia and Puerto Rico. 
Enclosed with the letter was a st. : tpecl , se L f-aidress-: j. post care 
that requested information regards.::; rhe r ^e of .h-. 3rtifica- 
tion Officer in that state, his or _.cial title, a. r^., and 
telephone number . Forty-three of the cards were xeru-^ned. The 
remaining names were arrived at by calling their State Boards 
of Education. 

Two interviewers divided the survey into twenty-si. x sites 
apiece. The interviewers were given an instruction sheet, that 
divided the states into time zones, so as to avoid inappropriate 
calling times. In addition^ further explanations of the eight 
parts of Questions 1 and 4 were provided. (The questionnaire for 
Puerto Rico was translated into Spanish by one of the inter- 
viewers.) Responses were limited to changes that had occurred 
from September^ 1969 to the present, and changes anticipated over 
the next two years. Response categories were Don't Know (DK) , 
No and Yes, with probing for each "Yes" response. A sonimc^ry 
sheet was kept on each state to record the number of calls (in- 
cluding busy signals) necessary to complete the questionnaire. 
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This Inf orjnation can he found witlx listiag of Director's najties 
and addresses in Section IV. 

When discussing changes, the intervievrer s asked the certi- 
fication officers about a specific set of eight changes. We 
were interested in whether changes in these areas had occurred in 
the past three years or were anticipated in the near future. 
These areas coi^cerned changes such as: 

a* Granting of provisional certification 

b. B.A. no longer required 

c. Number and/c type of courses requir--'d 

d. Credit given iior work experience outside of the 
teaching profession 

e. Credit given f^r hours of teaching beyond required 
practice teaching 

f. Exempti.;ns made for particular Local Education Agen-- 
cies (from state regul.: tions) 

g. Granting of special certificatim for teachers' aides 

h. Extensions m time requirements for certification 

Interviews were completed with 51 of 52 possible certification 
officers, the exception being Massachusetts whose officer, de- 
spite ten attempts, could not be reached. 

On the following pages are the Interviewer Instructions 
for the SEA Telephone Survey, 



In addition, the respondents were asked to specify any other 
changes which had been made or were planned. 



SEA TELEPHONE SURVEY 
INTERVIEWING INSTRUCTIONS 



2. Summary Sheet 



Schedule: O ir goal is to cor uct the survey i: all fifty (50) stat 

p': s Puerto Rico . .'U the IListrict c Columbia, over 
four day period. e schc;dule Vv'ili be as follows: 

Day 1 9:00 -11: 00 --all 1' - -la mma 2 

1 1:00-12:00— all 2' Alaska 4 

1:00- 3: 00 --all 3': i~ona 3 

3:00- 5: 00 --all -r'- Kansas 2 

C iliicrni" 4 

. / - ;. ,-iit 1 

iv 3 

Delaware 1 

D. C. 1 

Florida 1 

Georgia 1 

Hawaii 4 

Day 2 9:00- 11:00- -all I's Idaho 3 

ll:00-12:00--all 2's Iowa 2 

1:00- 4:00--an 2's Illinois 2 

4:00- 5:00--all 3's Indiana 2 

Kansas 2 

Kentucky 1 

Louisiana 2 

Maine 1 

Maryland 1 
Massachusetts 1 

Michigan .2 

Minnesota 2 

Mississippi 2 

Missouri 2 

Day 3 9: 00 - 1 2: 00 - -all I's Montana 3 

1:00- 2:00--all 2's Nebraska 3 

2:00- 3:00--all 3's Nevada 4 

3:00- 5:00--all 4's New Hampshire 1 

New Jersey 1 

New Mexico 4 

New York 1 
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Day 3, cont'd 
D ay 4 



Hours : 

Eligible 
Respondents : 

Quota: 

Summary 
Sheet: 



9:00^11:00--all ^ s 
11^00^12:00 

1: 0- 2:00--ali _ s 

2:00- 4:00-~all s 

4:0 0- 5: 00 --all -s 



N. Carolina 1 

N. Dakota 2 

Ohio 1 

Oklahoma 2 

Oreg^ " 4 

Penn .'.'vania 1 

Puerto r ico 1 

Rhode Is land 1 

S. Ca:"oi_na 1 

S. Dakr-n 3 

Te::as 2 

"vv c^snington 4 

W. Virginia 1 

Wisconsin 2 

Wyoming 3 



A total of seven (7) hours per day have been alloted for 
this task* Th^ preceding schedule was developed to 
accomodate the time change across the states, so as not 
to try to contact California at 7:00 a.m. , etc. 



The only respondents eligible to be interviewed are those 
whose names and phone numbers appear on the questionnaires. 

Each interviewer will be responsible for 25 states. 



A summary sheet has been attached to each questionnaire. 
The outcome of each dialing should be recorded on this 
sheet. 
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1.6 Satisfaction Checks 



Eight COP Directors Cone site per Abt staff) were contact- 
ed by the Abt Field C*:»ordinator and asked how they felt the test- 
ing went, if they had any problems with the Abt representative, 
and if they had any special considerations they wished to voice. 
Eight out of eight of the Directors reported a high degree of 
satisfaction with the field effort. 
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2-0 



Spring Survey Procedures 



The Spring field effort represented a continuation of the 
survey plans which called for revisiting sites and conducting 
post-tests^ new tests and additional interviews. Participants 
were po:^t-tested on the Adult Basic Learning Examination, Form B, 
in order to note any changes in basic skills over time and to re- 
la le such changes to data on demvOgraphics and/or opinions and 
attitudes towards the program. In addition, attitudes towards 
the program were related to job characteristics and current col- 
lege experiences. 

During this half of the evaluation, the impact on students 
was measured by achievement test scores and student opinions about 
schools, self and others. Standardized achievement test scores 
(reading and math subtests) were supplied by the sampled sites 
for students (grades I-VI) with COP aides and comparison stu- 
dents who did not have aides. In those sampled sites where 
achievement testing was not scheduled for the Spring, Abt Associ- 
ates Inc. tested the students using the Cooperative Primary Test 
(grades I-III) and the Comprehensive Test of Basic Skills (grades 
IV-VI) , reading and math subtests. In order to assess students' 
opinions about self, others and school, Abt Associates developed 
a "Student Opinion Instrument" (see Chapter 3 . 0 ) which was ad- 
ministered to all students in the sampled and comparison classes 
of the five student testing sites. 

In addition, institutional impacts were measured by type. 
Changes in classrooms, elementary schools, LEA standards, IHE 
courses and requirements were explored by questionnaires ^ each 
specifically designed for selected groups of respondents: teachers, 
principals, superintendents, college administrators and Project 
Directors. 
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2 . 1 Spring Field Prepara,tlons 



On 17 January 1972, the field staff member responsible 
for each of the sixteen sites contacted the Project Director to 
schedule the spring visit. The field staff member described to 
the Project Directors the purposes of the spring field effort and 
indicated that a letter fully outlining the spring field pro- 
cedures would be forthcoming • The conversation — including the 
agreed upon dates for the site visits — was confirmed in a letter 
sent to the Project Directors by Abt field staff members on 19 

A letter, dated 21 January was then sent to the Project 
Directors with a copy of the First Quarterly Report on COP. 
The purpose of the letter was to inform the Project Directors 
of the conduct of the evaluation up to that date (via the Quar- 
terly Report) , to describe in detail the spring survey needs at 
their site, and to outline the plans and logistical requirements 
for the spring visit to the sixteen sites. This letter outlined 
the following tasks: 

• Post-testing of those COP participants who took the 

ABLE test last fall. Testing would again take approxi- 
mately two and one-half hours and would include both 
the ABLE test and the Participant Questionnaire. (It 
was noted that the ABLE test would be a different form 
and that the questionnaire was a new one.) A request 
was made that the participants be tested in the same 
location as the fall visit, in instances where that 
was feasible. 

9 The distribution of the self -administered questionnaires 
to: the Superintendent of Schools, Principals of 
schools in which COP participants are assigned to teach, 
administrators/faculty of the college or university in 
which COP participants are enrolled (to be given to 
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those staff members most knowledgeable about the COP 
program and participants in relation to the college 
or university, possibly including instructors of COP 
courses^ counselors of COP students, coordinators of 
the COP program, etc) and the Project Director. 

• Since most of the field staff would not know the trans- 
portation routes between schools or other locations 
where questionnaires would be dropped off, it was re- 
quested that the Project Director accompany the staff 
member or, where this was not possible, provide the 

staff member with specific directions. It was recommended 
that the questionnaires be collected before the end 
of the business day, 

• Administration of post-tests to those COP participants 
^ho were unable to be present on the first day of test- 
ing. Post-testing would take place on Day Two of the 
field effort — time set as a contingency where tasks of 
Day One had not been completed. 

• Gathering of the self -administered questionnaires which 
were not picked up before the completion of the business 
day. Day One. 

The letter of January 21, 1972, also included the list of 
those COP participants- who were tested in the fall and who need- 
ed to be contacted for post-testing during the spring visit. It 
was requested that participants be encouraged to attend since new 
people, who had no base line data, could not be substituted. 

A list of school principals, superintendent of schools, 
and college or university faculty who would fill out self-admin- 
istered questionnaires was also included. These lists were pre- 
sented as a "double-check" to insure that there were no deletions 
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cr to encourage additions where the lists were incomplete. The 
Project Director was encouraged to acquaint the personnel on the 
enclosed list with the purposes of the visit, and to elicit their 
cooperation with the study during the forthcoming months. 

A reiteration of the guarantee of an honorarium to be paid 
to Project Directors was included in the letter of January. 21. It 
was noted that five Project Directors, at sites which included 
student testing, would receive $200; all other Project Directors 
would receive $100. Each staff member in charge of a site was 
given copies of this letter on. the 22nd of January. 

Ken Carlson contacted the five sites designated for stu- 
dent testing with the Student Opinion Instrument and the testing 
of teachers of both COP and non-COP classes in grades I-VI. 
These five sites were Hartford, Denver, Stockton, San Diego, and 
Asheville, North Carolina. The Asheville site also included 
student achievement testing, due to the fact that there is no 
standardized testing program in Asheville as in the other four 
sites. Achievement tests were given in the other four sites by 
their LEAs who agreed to send Abt Associates their results. 

Fiela staff members responsible for each site telephoned 
their Project Directors on February 1, 1972, to confirm that the 
letter of 21 January arrived and that all questions were answered. 
On March 3 the Field Coordinator wrote letters to all the Project 
Directors informing them of a change of handling procedure in. 
the Principal Questionnaire only . Because of the large number of 
principals involved in the spring effort, and becuase the field 
staff had only a short period in which to complete a myriad of 
tasks, it was decided that the Principal Questionnaires would be 
mailed directly to each Project Director (subsequent to 0MB clear- 
ance) . The self-administered questionnaires arrived, each with 
a principal's name attached, with instructions to the Project 
Director to call or contact the principal to acquaint him with 
the COP study before distributing the questionnaire. Principals 
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were also given a stajriped envelop addressed to Abt Associates so 
that they could mail the questionnaire directly baclc to the 
Field Coordinator. Each questionnaire was numbered by site and 
principal to maintain maximum in-house control. In addition^ the 
Project Directors were asked to inform the IHE personnel and 
superintendents of schools of the purposes of ov.r visit and to 
elicit their cooperation in filling out the self --administered 
questionnaire when the Abt field staff member arrived. Only in 
the case of the Cedar Rapids site (due to the fact that there 
were 22 LEAs , ) were the questionnaires for the superintendents 
handled in the same manner as the principal questionnaire for 
the other sites. 

Also included in the letter of March 3 ^.ras a list of all 
of those who would receive questionnaires at ee of the re- 
spective Project Director's sites. In the case of the five sites 
where student testing was to take place ^ an addendum was attached 
to the March 3 letter indicating that a complete list of those 
classes selected for student testing and those teachers who would 
be administered a questionnaire would be included in a forth- 
coming correspondence . 

On March 6, the Spring Field Guide was distributed by the 
Field Supervisor to all field staff involved in the spring effort, 
The guide was made available to the field staff prior to the 
first staff meeting for their perusal. On the following day, 
March 7, the Abt field staff met to review the spring field guide, 
Discussion of the meeting centered around securing travel arrange- 
ments, verifying field visit dates, reviewing the inscruments 
to be administered/ and clarifying and discussing any problems 
pertinent to the spring field effort. 

The Abt field staff member responsible for each site 
called Project Directors on March 10 to confirm the receipt of 
the letter sent on March 3. 
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2 . 2 Schedule for Spring Field Effort 



Date 



Event 



January 17 



Field staff contacted Project Director or liaison 
to schedule Spring visit at each site. Short de^ 
scription of the purposes of our visit followed by 
a letter describing in full the Spring field pro- 
cedures at each site. 



January 19 



Letter sent to Project Directors by Abt field staff 
to confirm dates for Spring visit. 



January 21 



Letter senb to 16 Project Directors detailing our 
plans for the spring visit. Participant lists , 
ncones of IHE personnel. Superintendents and Prin- 
cipals to respond to a ten minute self -administered 
questionnaire were included as well as the COP 
Pall Quarterly Report. 



February 1 



Field staff telephone Project Directors to check 
that letter arrived and that all questions were 
answered . 



March 3 Field Supervisor wrote letters to all Project Dir- 

ectors informing them of a change in the procedure 
for the Principals Questionnaire. The Principal 
Questionnaire will be mailed to Project Directors 
(subsequent to OMB clearance) with instructions 
to call/contact each Principal to acquaint him with 
the COP study before distributing the questionnaire. 
Principals will be given a stamped envelop addressed 
to Abt Associates so they can mail the questionnaire 
directly to us* Each questionnaire was numbered 
by site and Principal co maintain control in-house. 
In addition, the Project Director will be asked to 
inform the IHE personnel and Superintendents of the 
purposes of our visit and to elicit their coopera- 
tion in filling out the self -administered questionnaire. 
In sites where there are more than 2 or 3 LEAs the 
questionnaires were to be handled in the same manner 
as the Principal Questionnaire . 
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Date 



^ ent 



March 6 
March 7 



d Guide distributed to entire field staff. 



Abt field staff meeting held to review Spring Field 
Guide. 



March 10 



Staff called Project Directors as follow-up to a 
letter of March 3. 



March 13 



Letter sent to Project Directors of the five sites 
involving student testing. Letter included lists 
of classrooms and teachers selected for testing and 
a separate letter to be forwarded to teachers by- 
Project Directors outlining testing proce -^lures ■'.n 
teacher classrooms. 



March 15 Staff called Project Directors at each site to veri- 

fy that all sampled participants have been informed 
of the post-testing and that a room has been arranged 
for testing. Final arrangements to meet the Pro- 
ject Director/liaison at the site at least 15 minutes 
before testing were made. 

March 16 Staff made final travel arrangements to sites. 

March 20 Abt staff administering the Student Opinion In- 

strument received in-house training. 

March 23-24 All Abt field staff met to cover all phases of 

field effort. 

iMarch 30 Project Directors were called to verify final 

arrangements made on the 15th. 



April 3-21 All staff in field. Abt Associates Inc. was noti- 
fied daily as to- where staff could be reached. 

Return from Field Supervisor wrote letters to the Project Dir- 
field ectors to thank them for cooperating and to enclose 

the honorarium. 
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2.3 



General Field Staff Responsibilities 



•1 testing, except in th.e three largest sites (to 
be V. wsseu; , took place between 3 April and 21 April, At the 
eleven sites where no stude.nt testing was conducted, staff ad- 
ministered the ABLE test and the Participant Questionnaire to 
sampled COP participants in one day per site and, in most instances, 
distriiuted and collected the self -administered questionnaires 
the same day. A second day was slotted for use only as a con- 
tingency in cases where instruments could not be administered and 
data was not collected on the first day. As in the fall, field 
staff were responsible for scoring the ABLE tests. 

Except for the Principal's Questionnaire, which was handled 
separately (as described in the letter of March 3) , field staff 
members were responsible for distributing and collecting the 
appropriate number of questionnaires to IHE personnel, superinten- 
dents and Project Directors at each site. 

It was also the responsibility of each staff member to find 
a suitable time to test the COP participants with the ABLE and the 
Participant Questionnaire. Distribution of the IHE, Superinten- 
dent, and Project Director Questionnaires was scheduled around 
testing of COP participants — preferably with distribution in the 
morning and collection in the afternoon. Each respondent was to 
indicate the approximate time when the questionnaire would be 
ready for pick-up so that each staff member knew when he V70uld have 
all of the completed instruments. 

Before COP participants actually completed their self-ad- 
ministered questionnaire, the staff members at each site reacquainted 
them with the purposes of the study. The participants already 
had a fairly thorough overview of the study from the fall visit. 
However, it was explained: 



Why they (as a group) were being tested 

Why they were being tested a second time 

How the results were going to be \ised 

That we were not concerned with comparing parti- 
cipants against each other. 



1 
t 
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The purpose of the Student Opinion Instrument was to assess 
whether there was a difference in student attitudes in classes 
with COP aides as ^uipared to classes which did not have COP 
aides in grades I-VI. The instruments had been pre-tested and 
the final procedure for administering the instrument had been 
tested and revised three times. The procedures represented what 
Abt Associates felt to be the best way to present the test to 
the children. Field staff members were responsible for main- 
taining the stanaardized procedures for administering the test 
so that instructions given to students were identical across classes. 

2 . 4 Arrangements for Student Testing 

After securing lists of teachers who had classrooms chosen 
for administration of the Student Opinionnaire from the Project 
Directors, the Field Supervisor sent the Project Directors of 
these five . ites (Asheville, Hartford, San Diego, Stockton, and 
Denver) a letter explaining the procedures for administering 
the Opinionnaire. The letter to the- Directors requested that 
each Director notify the appropriate teachers that the Abt field 
staff would shortly be visiting his/her classroom. Enclosed with 
the letter to each Project Director was a letter to each of the 
teachers. It was requested that the Director forward these letters 
to each of the teachers having classrooms involved in student 
testing . 

The letter to the teacher briefly outlined the purpose of 
the study and explained that the Student Opinionnaire would be 
administered to measure attitudinal changes among pupils in class- 
rooms with and without COP aides. The field staff assigned to 
each site was introduced. In addition, a small amount of teacher 
preparation was requested : 

1) All students were to be arranged in rows; 

2) The students were to sit at least two feet from 
their left or right neighbor ; 
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3) Paper and materials would be supplied but the stu- 
dents should have their ovm pencils; 

4) The students vrere not to be told they were being t:ested, 
since this raises the level of anxiety which, would be 
unnecessary as the opinionnaire vras in a game'-like 
format. The teachers of classrooms with COP aides were 
also notified that Abt would like the teachers to fill 
out a self -administered questionnaire during the time 
that their students were being administered the opinion- 
naire. 

2 . 5 Spring Field Effort: New York, Detroit, Chicago 

Fall testing of participants was conducted in 16 sites 
chosen by the method of probability proportional to size (PPS) < 
This PPS sample was selected from a frame of COP projects re- 
stricted to those which had no fewer than 30 participants or no 
more than 20 0 participants. 

At the request of the Office of Education, the Spring 
testing was to include three extra sites that would insure a 
representation of COP participants from projects larger than 2 00, 
The three sites selected by the Office of Education were Brooklyn, 
New York; Chicago, Illinois; and Detroit, Michigan. 

Contact with the sites was initiated by Mr. Jerry Durley, 
Project Officer of COP, Washington, D.C. who spoke to the Pro- 
ject Directors and described both our company and the purpose of 
the study. Mr. Durley then sent us the names of the Project Dir- 
ectors who were expecting to hear from us. We in turn sent let- 
ters to each of the three projects, introducing ourselves and 
enclosing copies of our first Quarterly Report and a list of 
requirements for the Spring survey. 

Visits were subsequently scheduled to the sites, as follows 

Detroit: April 18-19 
New York: May 1-3 
Chicago: May 8-9 
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All three site visits were conducted by one Abt staff member who 
had established prior contact witPt tfie sites and vrho ^ 
dividually briefed by the Director of tbe Impact Evaluation aud 
the Field Coordinator. The field work proceeded very smoothly 
and information was obtained from the following respondents: 

• Project Directors 

« superintendents of Participating LEAs 

« Principals of Participating Schools in LEAs 

• college Administrators familiar with the COP program in 
their colleges 

• COP Participants (Aides) 

As in the case of the 16 other sites, the Principal Ques- 
tionnaires were mailed to the Project Director with instructions 
to distribute them to the Principals who would then return them 
to Abt Associates in the stamped Abt-addressed envelopes. Due 
to the large member of questionnaires distributed, ID numbers 
were not used. Although name and school were typed on labels 
on the questionnaire, the respondent was asked to detach this. 

The ABLE test was administered to the COP aides and 
scored by the field member before returning all forms via Emery 
Air Freight to the Cambridge office. The staff member reported 
that all three sites had been extremely cooperative and data 
collection had proceeded without difficulty. 

2 , 6 Honorarium 

An honorarium of $100 was mailed to 14 Project Directors 
accompanied by a letter thanking them for their extra asr^istar.ce 
and cooperation in carrying out the COP Impact Evaluation. The 
checks were sent on April 21 except in those cases where site 
visits were conducted at a later time; in these cases, checks 
were sent on April 27. Project Directors of the five sites where 
student testing had been conducted, received $200, since this ac- 
tivity required more extensive planning and assistance on their part 
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Field Guides 



This section is comDrised of iDOth the Fall and Spring 
Field Guides as they wer- presented to Abt field staff members. 
Both guides, although constructed for the most part by the Dir- 
ector of the Impact Evalaation and the Field Supervisor, were 
subject to revision and discussion by the Abt field staff. Sev- 
eral group meetings were held to dis-'uss the contents, proce- 
dures and schedules and the most efficient ana effective ways of 
fielding the study. Tra guides themselves include field sche- 
dules, responsibility matrices- and test instructions, as well as 
arrangements for distribution and collection of the self-admin- 
istered qvestionnaires. T-Taile in the field, each staff member 
was required to keep a cop;/ of -he appropriate guide closp. at 
hand at all times and to adhere to the Instructions within as 
closely as possible. 
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INTRODUCTION 

The purpose of this field effort is to collect soi^e important information 
on the participants in the Career Opportunities Program (COP) within the 
context of assessing the impact of COP on students, participants, and insti- 
tutions. 

The following materials* are included in this guide to acquaint you 
with the overall study: 

• IMPED Work Statement 

• Schedule for Fall Field Effort 

o Letter sent from Washington, D. C. to Project Directors 

• Letter sent from Abt Associates to Project Directors 

• List of Sites, Project Directors, Field Staff Assignments 

• COP Project Directors' Checklist 

The following are included for use in the field: 
Field Manual 

A* Instructions for Abt Associates Inc. staff 

1. Procedures for administering the Participant Questionnaire 

2. Procedures for administering the Adult Basic Learning 
Examination (ABLE) 

B. The Participant Questionnaire 

C. ABLE Handbook for test administration 

D. ABLE test booklet 

E. Scoring instructions 

All of the materials should be read before going out into the field. 
There will be a field staff meeting (date to be announced) to answer any 
questions you might have. However, do not hesitate to contact either Pat 
or Sydelle, at any time. 



J^Q^g' Parts of these materials are omitted from this section because 
they can be found in other parts of the quarterly report. 
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CODE: 

FIELD STAFF; 



SIGN -UP SHEET 



GRADE TEACHER'S NAME IN 

NAME . ASSIGNED ASSIGNED GR\DE 



3. 

5^ 
6. 



9^ 
10^ 
IL^ 
\Z, 
13^ 
14^ 
15^ 
16^ 
17^ 
18^ 
19. 



23. 
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GRADE TEACHER'S NAME IN 

NAME ASSIGNED ASSIGNED GRADE 

24. 

25. 



26. 
27. 
28. 
29. 
30. 
31. 
32. 
33. 
34. 
35. 
36. 
37. 
38. 
39. 
40. 
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FIELD MANUAL 
COP. FALL SURVEY 



A. INSTRUCTIONS FOR FIELD STAFF 

Materials 

This manual includes the following materials sequenced according to 
use in the field: 

1. Participant Questionnaire 

2. Adult Basic Learning Examination (ABLE) handbook for test 
administration. 

3. ABLE test booklet including Reading and Math subtests to be 
administered to COP participants. 

Schedule 

All field testing will be done on October 18, Field staff will administer 
the Participant Questionnaire and the ABLE tests within one day per site. 
Field staff will also score the tests for raw scores. 

Hours 

It is up to you to determine which hours are most convenient for the 
COP participants at your site. Many COP students will teach for part of the 
day and take courses at various places for the rest. However, in consultation 
with the COP Project Director at your site, arrive at a time which will be 
most suitable lor COF participants, as well as a testing location which is: 

1. easily accessible 

Zr. well-lit and relatively free of disturbances (street noise, othet 
students passing in the halls, etc. ). 

3, familiar to the students as to minimize discomfort or distraction 
about new surroundings. 

4. equipped with adequate desk or hard surfaces for writing 
Washington suggests that the college site be used.' 
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The following testing schedule represents the situation where COP 
participants are free for testing in the morning only: 

9:00-9:05 Liaisor * : irbduces Abt staff meinb<=ir 

9:05-9:15 Abt sta^ • ?7 v lYiber gives brief description of 

the task, &n ovtsrview of the project and the 
purpose of the testing(see page 5). 

9:15-9:30 Administration of Participant Questionnaire 

(15 minutes); collect questionnaire 

9:30-9:45 Instructions for ABLE Reading subtest 

9:45-10:42 Administration of ABLE Reading subtest 

10:42-.11:00 Short break 

11:00-11:10 Instructions for ABLE Arithmetic subtest 

11:10-12:00 Administration of ABLE Arithmetic subtest 

Part B: Problem Solving (50 minutes) 

12:00 Collect test booklets and thank participants 

for their cooperation 

The total testing time (not including instructions) is 2 hours and 30 
minutes. As indicated in your Handbook for Test Administration, the ABLE 
achievement tests include a time factor. Therefore, you must adhere as 
closely as possible to the time allotment for each test. The Participant 
Questionnaire is not a timed test, however. Therefore, you do not need to 
keep to a particvilar time schedule. 

Make sure you have the following materials in sufficient number for 
a,ll the participants: 

• pencils 

m Questionnaires 

• ABLE booklets and answer sheets 

• Sign and tape 

• stop watch or second hand 
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Procedures for Administering the Participant Questionnaire 



Before COP participants actually complete the Questionnair' , it is 
very important that they have a brief overview of what the study is ail about, 
particularly: 

• why they (the group you are testing are being tested 

• how the results are going to be used 

• that we are not concerned with comparing participants against 
each other but rather with the scores of the group as a whole to 
see if they have changed over the course of the year. Explain 
that we will test again in April. 

It is most important that the people being tested do not feel anxious about the 
fact that they, as individuals, are going to be singled out pr that the Project 
Director will find out about their scores and be disappointed, etc. We assume 
that there will already be some test anxiety among participants, as a result 
of bad testing experiences in public school. Therefore, every attempt should 
be made to make them feel at ease with you and with the testing situation at 
hand. 

When you read the Participant Questionnaire, you will sec. tr,at there 
is little room for misinterpretation. It has been constructed to be as straight - 
forward as possible, thereby making your task quite simple. 

Steps in Administering the Participant Questionnaire 

1. Introducig youfself and the study. Tke f oHr>^wing paT^gt?a:ph Twl 

be useful in deciding what to say: 

*'Hi. My name is and I'm from Abt Associates Inc. , a research 

firm in Cambridge, Mass. We are presently working with the Office of 
Education to see what kind of impact the COP program has had on you as 
participants, the students you teach, and the colleges you attend. My purpose 
here today is to gather some information about you in two ways: first, 
through a questionnaire which you will be answering and second, through a 
reading and an arithmetic test. I want to emphasize, though, that none of 
these tests will be used or even seen by anyone but Abt Associates staff. 
In other words, none of your professors or teachers and not even Mr. 

(the Project Director) will ever find out how you did on these 

tests, as i ndividuals . In other words, no one will ever see a test with a name 
on it. If a test has to be used by someone who knows you personally, we^ll 
cross your name off of it first. I want to make sure you understand this. 
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This kind of evaluation is required by the heads of COP in Washington, C. 
They want to know how well you are doing as a group. The results of these 
tests are very important to the future of the program- -so please bear with 
us. We will come back to test you again in April, and by looking at the scores 
from today and the scores in April we hope to reach some conclusions about 
the COP program. Would anyone like to ask any questions before we begin? " 

2. Hand out Questionnaire 

3. Ask participants to write the first initial and last name in the 
upper right hand corner of the Questionnaire. 

4. Ask them to complete each question and raise their hands if 
at any point there is an unclear question. 

5. Begin test. 

6. End test when all have completed the Questionnaire. 

7. Ask participants to make sure that their names are in the upper 
right hand corner of the test. 

8. Collect and put away. 

2. Procedures for Administering the ABLE Tests 

Although the ABLE includes tests of Vocabulary, Spelling, Reading, 
Arithmetic Computation and Arithmetic Problem Solving,for the purpose of 
this study we wi)l only use the Reading Test (both Paragraphs and Newspaper) 
and the Arithmetic Problem Solving Test. You will notice that the test 
booklet is stapled closed from pp. 1-8 and from pp. 19-24 to minimize confusion 
about which pages to fill, which test is being taken, etc. 

The ABLE Handbook for Test Administration provides all the instructions 
and special considerations you will need to know about and ten participatnts- 
during test administration. It is important --albeit, painful-^to say the directions 
(the darkened lettering) just as they have been written. Although they are by 
no means written with style or even humor, they cover all the salient points 
which need to be mentioned and are an integral part of test reliability. 

Be sure to: 

«> speak loudly enough so that everyone can hear you. 

• read the instructions slowly and encourage questions. 

• if any directions seem ambiguous to you, be sure to explain 
it in full in your own words. 

• try as mu'jh as possible to maintain eye contact 
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steps in Adminstering the ABLE Ter ts 



1. Post a sign on the door reading, TESTING, DO NOT IISTURB^' (prepare 
one and bring thumb tacks or tape). 

2. Be sure to have your own blank copy of the test booklet and answer sheet 
to use for demonstration. 

3. Remind participants that you are not concerned with individual scores on 
the two tests they will be taking, the Reading and Math tests. You are 
interested in the total scores of the group. The tests will not be seen or 
be used by anyone else but Abt Associates staff. Anyone else WILL NOT 
SEE THEIR TESTS with their names on them. If any individual tests are 
seen, the names will be crossed off. 

4. Distribute pencils and tests, reminding participants not to open them 
imtil instructed to do so. 

5. Explain that the first test will be a Reading test. 

6. Begin instructions. 

7. Display the answer sheet or test booklet as much as possible- -the more 
the better. People often have difficulty finding pages, numbers on the 
page, etc. 

8. Begin test. Reading Test: Paragraph Section 

Start: 9:45 
Finish: 10:42 

Time left: 23 minutes (post time every 10 minutes) 

9. End Paragraph section EXACTLY FORTY MINUTES FROM THE START 
OF TESTING. 

10. Read instructions for Newspaper section. 

11. Begin Newspaper section (note timed reading in this section). 

12. End Newspaper sections EXACTLY SEVENTEEN MINUTES FROM^THE 
START OF TESTING. 

13. Collect materials. 

14. Take a short break. 

15. Re-distribute materials when participants are seated. 

16. Read instructions for Arithmetic test (#4: Part Be Problem Solving). 

17. Begin test. 

18. End test EXACTLY FIFTY MINUTES FROM START OF TESTING 

19. Collect materials. 

20. Thank group for their help. 
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Scoring Instructions 

Abt Associates field staff will be responsible for scoring the two 
ABLE subtests - -not the Participant Questionnaire. The COP participants 
will record their answers on ar. IBM sheet (not in the test booklets) which is 
scored quite easily by placing answer key (which will be provided) directly 
on top of the answer sheet and marking the slots with a colored pencil. Raw 
scores for each subtest are arrived at by simply counting the number of 
correct responses and recording them at the bottom of each participant's 
IBM card. 
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COP FIEL - ^ XK i 

TO BE COMP: /TED EACH SITE VISIT 



Before test ig- - i o yo:. have each of the f oll owing : 

1. "'^z. ::ign-.up sheet for participant's names, grade assignnients, 

.n(.. teacher in assigned grade? 

2. --gn reading "Testing, Do Not Disturb'^ 

3. ~0 pencils. 

4. Checked to see that all IBM cards have: 

--the site code number on upper right hand corner 

--space for participants to write their birth date instead of 
their names. 

5. Checked to see that the Participant Que stionnaires have: 

--site code number on top right hand corner 

--place to record math and reading scores on bottom of last 
page 

(REMEMBER TO- RECORD THESE SCORES ON THE PARTI- 
CIPANT QUESTIONNAIRE AFTER YOU HAVE CORRECTED 
THE ABLE TESTS! ) 

It is most important that before you begin to test you read the para- 
graph explaining the study and why we are testing COP participants. Also, 
remember to explain that we are asking for names on the sign-up sheet 
ONLY BECAUSE WE MUST HAVE A RECORD FOR SPRING TESTING, 
Mako sure everyone understand? tnac we have no way of matching birth dates 
(which is the only form of identification on the tests) with their names. 

After Adminis tration of the Participant Questionnai r e r 

Collect all questionnaires. During administration of the first ABLE 
test, check to make sure that no two people have the same birth dates (on the 
participant questionnaire). This is a most important step. If two or more 
people have t' e saiae date of birth, ask them tc come up to you and to put 
their social sr-^curit-^ numbers (or if they are uiLavailable, their license 
number?^ or s nme fr rm of identification to distinguish them) next to their 
birth dat'^'s so ihat > e can tell them apart. 
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After Testing is Conapleted; 

When each ABLE test has been scored withthe Answer Key supplied 
to you. please record the ABLE Math and Reading subtests scores on the 
corresponding Participant Questionnaire for that person. Space for record- 
ing these two scores is on the last page, bottom ri^t hand corner. 
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FIELD GUIDE 
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INTRODUCTION 



BEFORE YOU GO ANY FURTHER, PLEASE REREAD COMPLETELY THE 
COP FALL FIELD MANUAL. 

The purpose of this effort is to: 

1. Retest COP participants utilizing alternate Form B of the ABLE 
test and administer a more extensive Participant Questionnaire 
than that administered in the fall. 

2. Distribute and collect self -administered questionnaires to IHE 
personnel, superintendents, and Project Directors. 

3. Test children in both COP and non COP classes, grades I-VI, 
utilizing the Student Opinion Instrument. This procedure will 
only be carried out in five of the sixteen sites: Hartford, 
Asheville, Stockton, San Diego and Denver. Also distribute and 
collect self-administered questionnaires from the teachers of 
the tested classes at these sites. 

4. Conduct achievement testing of students in grades I-VI at the 
Asheville site only. 

The following materials are included in this guide: 

Field Guide 

1. Map of sites and site lists 

2. Updated list of Project Directors 

3. Communication with sites since the Fall field effort 
4.. Schedule for Spring field effort 

5. Staff Responsibility Matrix 

The .following are included for use in the field: 
Fielc Manual 

Instructions for Field Staff 

1. Administration of the Participant Questionnaire 

2. ABLE test administration and scoring instructions 

3. ABLE Handbook for test administration 

4. Procedures for administering questionnaires to principals, 
IHE personnel. Superintendents and Project Directors 

5. Procedures for administering the Student Opinion Instrument 
(Instructions for student achievement testing will be handled 
separately. ) 

6. Procedures for administering questionnaires to teachers 
(in the five student sites) 
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1. COP SITES: RANDOM SAMPLE BY PPS 
(Approved by Program Director) 



Numtfer 




# 


Listtf 


Site 


TT / 

Urban/ 
Rural 


Participant 

# 






1 


7 


1 7 


San Diego, Cal, 


TT *L.^ 


155 






7 




1 A 
1 D 


riartiora. Conn. 




97 




^ i UD 


-a 


c 


3 / 


Lakewood, CoL 


R 


63 




1 on 


A 
*r 


0 


i 3 


irtockton, Cal. 


u 


66 




UVoV 




/7T5 \ A 


ic 


oan jJiego, Cal, 


( ) 


166 








7 
1 




jrt.sneviiie, in, ^» 


TT J^'D 


75 




1 ^ I? I? 






1 o 


riartiora, conn. 




97 




6119 


/ 

D 


8 


87 


Charleston, W. V. 


R 


72 




1547 


7 


9 


20 


Jacksonville, Fla. 


U 


133 




4248 


8 


10 


55 


Las Cruces, N. M. 


U 


30 




2352 


9 


11 


29 


Cedar Rapids, lo. 


U+R 


58 




0449 


10 


12 


6 ' 


Los Angeles, Cal. 




160 




3596 


11 


14 


44 


Jackson, Miss. 


u 


106 


i 


5980 


12 


15 


85 


Spokane, Wash. 


u 


30 




4605 


13 


16 


62 


Belcourt, N. D. 


R 


59 


% 


1956 


14 


18 


24 


Lafayette, Ga, 


R 


48 




4515 


15 


19 


60 


Lillington, N. C. 


R 


57 




2866 


16 


21 


34 


New Orleans, La. 


R 


30 


i 



*R=one repeat 
**=Model Cities 
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2. COP PROJECT DIRECTORS 



Mr. G. Worth Booth 
Asheville City Schools 
Box 7347 

Ashville, North Carolina 
(704) 254-3395 

Mrs. Ophelia Boswell 
Stockton Unified School District 
701 North Madison 
Stockton, California 



Mr. William Davis 
Joint Co'Jinty, School System 
4401 6th Street, S. W, 
Cedar Rapids, Iowa 52401 
(319) 366-7601 

Mrs. Mary Heinkel 

San Diego City Schools 

4100 Normal Street 

San Diego, California 92103 



Mr. Kenneth Harrison 
7900 Washington Avenue 
Xavier University 
New Orleans, Louisiana 
(504) 486-7428 

Mr. Richard H. Hunt 
Kanawha County Board of Ed. 
5445 Big Tyler Road 
Charlestown, West Virginia 25434 
(304) 776-3370 

Mr. William Jones 

Los Angeles Unified School District 

Box 3307 

Los AngQles, California 90054 



Mr. John Dunn 
Harnett County Schools 
Box 1028 

Lillington, North Carolina 
(919) 893^5161 



Dr. Dale Pennybaker 

Jefferson County School District 

809 Quail 

Lake wo od^ Colorado 
(303) 237-6971 

Dr. Vito Perrone 
New School of Behavioral Studies 
University of North Dakota 
Grand Forks, North Dakota 
(701) 777-2361 

Mr. Don Summers 

Career Opportunities Program 

500 Woodland Street 

Hartford, Connecticut 

(203) 566-6034 

Mr. John F. Stolz 
Spokane School District iSl 
West 825 Trent Avenue 
Spokane, Washington 99201 
(509) 938-8581 

Mr. J. Paul Taylor 

Las Cruces School District 

301 West Amador Avenue 

Las Cruces, New Mexico 88001 

(505) 524-2894 

Mr. Richard Wieczorek 
Walker Court 
South Duke Street 
Lafayette, Georgia 3Q729 
(404) 638^1240 

Mr. Joseph Williams 
Duval Cojinty School Board 
1011 Gilmore Street 
Jacksonville, Florida 
(904) 398-8331 

Mr. Walter E. Carr 
Jackson Municipal Separate 

School District 
P. O. Box 2338 
Jackson, Mississippi 
(601) 353-3095 
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5. A STAFF RESPONSIBILITY MATRIX 



Site 


Staff 


Dates 


Responsibilities 


! 

New Mexico 


W. Abt 


Thurs. 4/6- 
Fri. 4/7 


1. Test sampled COP 
participants with ABLE 
and Participant Question- 
naire, 

2. Distribute IHE, Super- 
intendent, Project Director 
Que s tionnair e s . 

3. Collect above 


New Orleans 


J. Beckmann 


Mon. 4/3- 
Tues. 4/4 


Same as New Mexico 


Jackson, 
Mississippi 


J. Beckmann 


Wed. 4/5- 
Thurs, 4/6 


Same as New Mexico 


Cedar Rapids 


J, Doucette 


Mon. 4/lO 
Tues. 4/11 


Same as New Mexico 


Jacksonville, 
Florida 


K, Carlson 


Mon. 4/3- 
Tues. 4/4 


Same as New Mexico 


Lafayette, 
Georgia 


K, Carlson 


Thurs. 4/6- 
Fri. 4/7 


Same as New Mexico 


N. Dakota 


R. Milnes 
D. Geller 


Mon. 4/17- 
Tues. 4/18 


Milnes: 

1. Test sampled COP 
participants with ABLE 
and Participant Question- 
naire at Fort Totten, 
Belcourt. 

2. Distribute and collect 
Superintendent Question- 
naire. 

Geller: 

1. Test sampled COP 
participants with ABLE 
and Participant Question- 
naire at Grand Forks. 

2. Distribute and collect 
IHE, Project Director 
Questionnaires. 



Staff Responsibility Matrix Cont'd 



Site 


otaii 


Dates 


Responsibilities 


Spokane 


R, Milnes 


Thurs. 4/:?0- 
r ri, ^ / Lfl 


Same as New Mexico 


^ 

Los Angeles 


J, Butenas 


Sat. 4/8 


Same as New Mexico 


Charleston, 
West Virginia 


L, Studen 


Mon 4/17- 
Tues. 4/18 


Same as New Mexico 


Lillington 


D. Geller 


Mon. 4/10- 
Tues. 4/11 


Same as New Mexico 


BITES WITH SI 


:UDENT TESTING 


Cook: 

1, Administer ABLE and 
Participant Questionnaire, 
Z, Distribute, collect IHE, 
Project Director Ques- 
tionnaires, 

3 , Assist in admini st ration 
of Student Opinion Instru- 
ment(SOI), 
Hailey: 

1, Administer SOI to four 
classes in grade II. 
Studen: 

1, Administer SOI to four 
classes in grade 1. 


Hartford 


P. Cook- 
L. Hailey 
L. Studen 


Mon. 4/3- 
Tues. 4/4 


Denver 


W. Abt^:^ 
D. Sadigur 
Li, Stuart 


Mon. 4/3- 
Tues. 4/4 


Abt: 

1, Administer ABLE and 
Participant Questionnaire. 
Z. Distribute, collect IHE, 
Project Director Ques- 
tionnaires. 

3, Assist in administration 
of Student Opinion Instru- 
ment (SOI). 
Sadigur: 

1, Administer SOI to four 
classes grade IV; two 
classes ^rade V. 
Stuart: 

1, Administer SOI to four 
classes grade VI; two 
classes grade V.-^'^--'^ 



=:= Indicates Director. 

Administer Teacher Questionnaires. 
One experimental; one control. 
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Staff Responsibility Matrix, Cont'd. 
Silres with Student Testing 



Site 


Staff 


Dates 


Responsibilities 


Stockton 


J, Butenas^!^ 
D, Sadigur 
Lr, Stuart 


Thurs, 4/6- 
Fri, 4/7 


Butenas: 

1, Administer ABLE and 
Participant Questionnaire, 

2, Distribute, collect IHE, 
Project Director Oues- 
tionnaires, 

3, Assist in administration 
of SOI, 

Sadigur: '''''^ 

1. Administer SOI to four 
classes grade I; two 
classes grade IL 
Stuart: >;c;ic 

1. Administer SOI to four 
classes grade VI; two 
classes grade V, 


San Diego 


J, Butenas'l' 
D, Sadigur 
L,, Stuart 


Mon, 4/10- 
Tues, 4/11 


Butenas: 

1, Administer ABLE and 
Participant Questionnaire. 

2, Distribute, collect IHE, 
Project Director Ques- 
tionnaires, 

3, Assist in administration 
of Student Opinion Instru- 
ment (SOI), 

Sadigur: 

1, Administer SOI to four 
classes grade I; four 
classes grade II, 
Stuart: 

1, Administer SOI to four 
classes grade III; two 
classes grade IV, 


Asheville 


K, Carlson-I' 
Lr. Hailey 
Lc Studen 


Mon. 4/10- 
Fri, 4/14 


To Be Supplied 



Indicates Director 



Administer Teacher Questionnaires. 

I 



ERIC 



134 



FIELD MANUAL 
COP SPRING SURVEY 



INSTRUCTIONS FOR FIELD STAFF 
Materials 

This manual includes the following materials sequenced according to use in 
the field: 

1. Participant Questionnaire 

2. Adult Basic Learning Examination (ABLE) handbook for test 
administration. 

3. ABLE test booklet including Reading and Math subtest to be 
administered to COP participants. 

4. Questionnaires to be self -administered to IHE personnel (COP 
counselors and instructors, admissions officers), Superintendents 
and Project Directors. 

5. The Student Opinion Instrument (for administration in five sites 
only) . 

Schedule 

All field testing will take place betvVeen April 3 and April 21. At 
the eleven sites where there will be no student testing, field staff are ex- 
pected to administer the ABLE test and the Participant Questionnaire to 
sampled COP participants in one day per site, and in most instances to 
distribute and collect the self -administered questionnaires on the first day 
as well. The second day is to be used only as a contingency in cases where 
instruments could not be administered and data was not collected on the 
first day. As in the fall, field staff will be responsible for scoring the 
ABLE tests. 

Hours 

It is up to you to determine which hours are most convenient for 
testing the COP participants at your site(s). In consultation with the COP 
Project Director, arrive at a time which will be most suitable for COP par- 
ticipants and if at all possible use the same location for testing which was 



used in the fall. 



The following sample schedule for testing with the Participant Ques- 
tioniiaire and the ABLE represent, the situation where COP participants are 
free for testing in the morning only: 

9:00- 9:05 Liaison man/Project Director introduces Abt staff 

member 

9:05- 9:15 Abt staff member gives brief description of the task, 

an overview of the project and purposes of testing 
(see page 

9:15- 9:30 Administration of Participant Questionnaire (15 min- 

utes); collect questionnaires 

9:30- 9:45 Instructions for ABLE reading subtest 

9:45-10:42 Administration of ABLE reading subtest 

10:42-11:00 Short break 

11:00-11:10 Instructions for ABLE arithmetic subtest 

11:10-12:00 Administration of ABLE arithmetic subtest Part B: 

Problem Solving (50 minutes) 

12:00 Collect test booklets and thank participants for their 

cooperation. Assure them that they can receive 
their test scores by writing directly to you as before* 

The total testing time (not including instructions) is 2 hours and 30 
minutes. As indicated in your Handbook for Test Administration, the ABLE 
achievement tests include a time factor. Therefore, you must adhere as 
closely as possible to the time allotment for each test. The Participant 
Questionnaire is not a timed test, however. Therefore, you do not need to 
keep to a timed schedule. 

Make sure that you have the following materials in sufficient numbers 
for all the participants: 

• Stop watch or second hand 

• Sign reading "Testing, Do Not Disturb*' and tape to put it up on 
the door . 

• 40 pencils 

• Checked to see that all IBM answer sheets have 

o the site code number on the upper right hand corner 
• space for participants to write their birth datf ^ instead of their 
names 
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• Checked to see that the Participant Questionnaires have; 

• the site code number on top right hand corner 

• place to record math and reading scores on bottom o£ last page 
(REMEMBER TO RECORD THESE SCORES ON THE PARTICI- 
PANT QUESTIONNAIRE AFTER YOU HAVE CORRECTED THE 
ABLE TESTS!) 

1 , Administration of the Participant Questionnaire 

Before COP participants actually complete the questionnaire, it is 
very important that you reacquaint them with the purposes of the study. The 
participants should already have a fairly thorough overview of what the study 
is about from your fall visit. However, you should not assume that all par- 
ticipants recall all the details you previously covered. Therefore, go over 
the main points listed below making sure to explain things clearly and to 
elicit questions when you have finished. 

• Why they ( the group you are testing) are being tested; 

• Why they are being tested for a second time; 

• How the results are going to 'be used; and 

• That we are not concerned with comparing participants against 
each other. • We are interested in the scores of the group as a 
whole to see if they have changed over the course of the year. 
Explain that this will be the last testing. 

Be certain to establish that this testing session is almost identical to the 
4 one in the Fall. It is most important that the particir *nts do not feel 

anxious about the fact that they are being tested. You should remind the 
participants that the Project Director will not find out about their scores and 
that no one but Abt staff will see individual tests.. We assume that there 
will already be some test anxiety among participants as a result of bad 
testing experiences in public school. Therefore, every attempt should be 
made to make them feel at ease with you and the testing situation at hand. 

When you read the Participant Questionnaire you will note that there 
is little room for misinterpretation. It has been constructed to be as straight- 
forward as possible, thereby making your task quite simple. 
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steps in AdTninistering the Participant Questionnaire 

*• 

1. Introduce yourself and the study. The following paragraph might be use- ^ 

■ * 

ful in deciding what to say: 

"Hi! My name is , I»m from Abt Associates ^ 

Inc. , a research firm in Cambridge, Mass. Some of you may 
remember me from my visit here last fall. As you may remem- ... 
ber, we are presently working with the Office of Education to ' '\ 

see what. kind of impact the COP program has had on you as par- 
ticipants, the students you teach, and the colleges you attend. 
My purpose here today is to gather the same kinds of information T 
about you that I did in October. I will give you a questionnaire L 
to answer and a reading and math test. I want to emphasize, 

though, that none of these tests have been or will be used by : 
anyone but the Abt Associates staff. In other workds, none of \ 
your professors or teachers, and not even Mr. 
(the Project Director) will ever find out how you did on these 

tests or the tests given in October. No one will ever see a test \ 
with a name on it. If a test has to be used by someone who knows i 
you personally, they will have no way of knowing which test is 
yours. I want to make sure you understand this. 

"As I mentioned during the test you took in October, this kind of 

evaluation is required by the heads of COP in Washington, D. C. 

They want to know hcv ^ell you are doing as a group , i^Ow as / 

individ-Zials.. The resj.lxs of these tests are very important to 

the fuL-zi— of the program- -so please bear with us for* this final 

testing. Sy looking at the scores' from October and the scores j 

we gei today, we hope to reach some conclusions about the COP 

prograiT^. Would anyone like to ask any questions be: re we 

begin * c- 
Hand out Cui: - :ionnaire. 

Ask the ptrr :r ipants to WTite the date of their bixth in the upper right {" 
hand corner . Indicate that they should write the month fir^t, day 
second and /^ar last. 

4. Ask them to complete each question and raise their hands if there is an 
unclear question at any point. 

5. Begin test. 

6. End test when everyone has completed the Questionnaire. 
7o Ask participants to make sure that their birthdates are in the upper 

right hand corner of the test. 
8. Collect the Questionnaires and put them away. 
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After Administration of the Participant Questionnaire 

Collect all the questionnaires. During the administration of the first 
ABLE test, check to make sure that no two people have the same birthdates 
(on the Participant Questionnaire). This is a most important step. If two 
or more people have the same date of birth, ask them to come up to you and 
to put their social security numbers (or if they are unavailable, their license 
numbers or some form of identification to distinguish them) next to their 
birth dates so that we can tell them apart. 

2' Administration of the ABLE Test and Scoring Instructions 

Although the ABLE includes tests of Vocabulary, Spelling, Reading, 
Arithmetic Compuation and Arithmetic Problem Sovling, for the purpose of 
this study we will only use the Reading Test (both Paragraphs and Newspaper) 
and the Arithmetic Problem Solving Test, You will notice that the test 
booklet is stapled closed from pp. 1-8 and from pp. 19-24 to minimize con- 
fusion about which pages to fill, which test is being taken, etc. 

The ABLE Handbook for Test Administration provides all the instruc- 
tions and special considerations you will need to know about and tell partici- 
pants during test administration. It is important- -albeit, painful --to say 
the directions (the darkened lettering) just as they have been written. 
Although they are by no means written with style or even humor, they cover 
all the salient points which need to be mentioned and are an integral part of 
test reliability. 

Be sure to: 

• speak loudly enough so that everyone can hear you; 

• read the instructions slowly and encourage questions; 

• if any directions seem ambiguous to you be sure to explain 
it in full in your own words; and 

• try as much as possible to maintain eye contact. 
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Steps in Adminstering the ABLE Tests 



!• Post a sign on the door reading, ^^.TESTING, DO NOT DISTURB'' (prepare 
one and bring thumb tacks or tape), 

2, Be sure to have your own blank copy of the test booklet and answer sheet 
to use for demonstration, 

3, Remmd participants that you are not concernec with individual scores on 
the two tests they will be taking, the Reading azid Math tests. You are 
interested in the total scores of the group. The tests will not be seen or 
be used by anyone else but Abt Associates staff. Anyone else WILL NOT 
SEE THEIR TESTS with their nancies on thenn. If any individual tests are 
seen, the names will be crossed off. 

4, Distribute pencils and tests, reminding participants not to open them 
until instructed to do so. 

5, Explain that the first test will be a Reading test, 

6, Begin instructions.. 

7, Display the answer sheet or test booklet as much as possible ---the more 
the better. People often have difficulty finding pages, numbers on the 
page, etc. 

8, Begin test- Reading Test: Paragraph Section 

Start: 9:45 
Finish: 10:42 

Time left: 23 minutes (post time every 10 minutes) 

9, End Paragraph section EXACTLY FORTY MLNUTES FROM THE START 
OF TESTING. 

10. R.ead instructions for Newspaper section. 

11. BegLn Newspaper section (note timed readirgrin this section). 

12. End Newspaper ^^^ctions EXACTLY SEVENTEEN MINUTES FROM THE 
START OF TESTING. 

13. Collect materials. 

14. Take a short break. 

15. Re -distribute materials when participants are seated. 

16. Read instructions for Arithmetic test (#4: Part Be Problem Solving). 

17. Begin test. 

18. End test* EXACTLY FIFTY MINUTES FROM START OF TESTING 

19. Collect materials. 

20. Thank group for their help. 

Staff may use "watch-restart" method in giving the ABLE, i. e. , start watch 
on the hour for each section. This is an easy way to keep time. 
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Scoring Instructions 

Abt Associates field staff will be responsible for scoring the two 
ABLE subtests--not the Participant Questionnaire. The. COP participants 
will record their answers on an IBM sheet (not in the test booklets) which is 
scored quite easily by placing an answer key (which will be provided) directly 
on top of the answer sheet and marking the slots with a color ed. pencil. Raw 
scores for each subtest are arrived at by simply counting the number of 
correct responses and recording tixem at the bottom of each participant's 
IBM sheet. 

When each ABLE test has been scored with the Answer Key supplied 
to you, please record the ABLE Math and Reading subtest scores on the 
corresponding Participant Questionnaire for that person. Space for recording 
these two scores is on the last page, bottom right hand corner. 

3.. Administration of the Teacher Questionnaire 



The self-administered Teacher Questionnaire will be given only to 
those teachers in whose classrooms the Student Opinion Instrument is fielded 
The questionnaire, which takes approximately 10 minutes to com.plete, will be 
given to the teachers while their students are beine tested 
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Level III 



DESCRIPTION OF THE EXAMINATION 

The AdvZt Basic Learning Examination (ABLE) is a batter;' of 
tests desired to measure the level of educational achievement amtrn^ 
adults. Although the examination is designed for use with adults ^ 
consists of items with adult content, it maj'- be used to assess achieve 
ment as low as primary-grade le^^el. ABLE was developed to fill . - 
need for an instrument to determine the general educational leroi 
adults who have not completed formal twelfth-grade education, as 
as for use in evaluating efforts to raise the educational level of snz 
adults. 

The examination was developed in consultation with a varie™ r 
people working with the many facets of adult education, and was i^vrji 
ten in accordance with the following general guidelines : 

1. Test content which is perceived as meaningful and useful 
adults, and which is meaningful from the standpoint of currix^^ 
ular validity 

2. Measurement of achievement from the primary grades thron:r ; 
the secondary level 

3. Realistic format which makes test taking more acceptable t 
the adult. 

4. Coverage of only the fundamental areas, thus keeping over^. 
testing time as short as possible 

5. Ease of administration for teachers with relatively limited e::^ 
perience in testing 

In order to cover as much as twelve years of school achievement, 
it was necessary to develop three batteries: Level I (Grades 1-4), Level 
II (Grades 5-8 and Level III (Grades 9-12). At each level, ABLE 
includes the following tests: 

Vocabulary Reading Spelling Arithmetic 

At each level, there are two forms available, Form A and Form B, 
which are comparable in content, format, and level of difficulty. These 
two forms may be alternated when re-evaluation or periodic testing 
is desired. 

All tests at Levels I and II are essentially without time limits. At 
Level III, all tests (except Vocabulary, which is dictated) are time- 
liniit tests; the time limits, however, are provided for administrative 
convenience rather than for fhe purpose of placing any premium upon 
speed. The time limits in all cases are generous, and are calculated to 
give practically all students sufficient time to attempt all questions 
which they are capable of answering. Thus, the Adult Bamc Leami7i::i 
Examination f ABLE ) , at all levels, may be considered as fundame:?- 
tally a power test. 
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ABLE will reveal the relative strengths and weaknesses of an in- 
dividual from one content area to another, but it was not designed to 
diagnose in depth what specific problems a student has within each 
subject area. For example, it might be revealed that a student's achieve- 
ment level is considerably lower in arithmetic than in other areas meas- 
ured, but it cannot be determined if this i? due to a lack of understand- 
ing of multiplication of fractions, long division, placement of decimal 
points, etc. 

The response mode of ABLE Levels I and II varies with each par- 
ticular test because the multiple-choice type of item was iiot always 
considered suitable for adults at these levels. At Level III, all questions 
are designed in the multiple-choice format and separate answer sheets 
are used, since adults who have reached that level of i^cholastic achieve- 
ment are generally capable of handling such a format. 

Descriptions of the purposes and characteristics of each test at 
Level III follow. 



Reading 

The reading test (60 items) determines how well the student can 
understand paragraphs and news items which he reads. The reading 
material contains a great variety of content, with heavy emphasis on 
the everyday life of adults. The first part of the test ( 45 items) includes 
paragraphs of varying length and complexity. Each paragraph is fol- 
lowed by a series of multiple-choice questions designed to test the stu- 
dent's ability not only to comprehend what is explicit in the matenal 
but also to make inferences and draw generalizations from what is 
given. The second part of the test is a newspaper reading section. The 
student is given five minutes to read a facsimile of a newspaper's front 
page. He is then required to answer 15 multiple-choice questions which 
measure his ability both to recall specific information and to form con- 
clusions from what he has read. The test format prevents the student's 
rereading the news items while he is answ"ering the questions. 



144 



Level III 



Arithmetic 

To obtain valid measurement of the student's arithmetic ability, 
ABLE includes separate" arithmetic computation and arithmetic prob- 
lem solving subtests. Although the tv/o are administered and scored 
separately, the scores are usually combined for a total arithmetic score. 
Together the subtests determine the overall arithmetic achievement of 
each student and give an indication of the relative strength of the tvt^o 
areas sampled. 

Arithmetic computation (Part A) includes 42 multiple-choice 
arithmetic problems testing basic numerical competence. Ari thmetic 
problem solving (Part R ) tf^^t<^ pv^ot\r^i] pnplira tion of these computa - 
tions in the solution of 42 realistic adult problems. The vocabulary 
level has been kept below that of the readmg^test. ' 



GENERAL DIRECTIONS FOR ADMINISTERING 

The examiner should become thoroujrhiy familiar with all the 
following directions before attempting in administer the exam- 
ination. It is siiggrested that, if possible, the examiner first 
"administer" the test to another adult ( for example, to a colleague 
on the staff) in order to make sure the procedures to be used are 
clear in his mind. 

1. Before beginning any test, the examiner should see that each stu- 
dent has adequate, welWi^^hted work space and that he has one or two 
sharpened pencils and a good eraser. A supply of extra pencils should 
be on hand. Scratch paper should be available for use in the arithme- 
tic tept. 

2. When administering the test, the examiner should have a blank copy 
of the test booklet and ot the answer sheet to be used for demonstration, 
along with a copy of this HandbooK. 

3. A normal class atmosphere should be maintained insofar as that is 
possible. Evevy effort should be made to present the testing procedure 
in a non -threatening manner. 

4. Students should be seated in such ?« way that they can all hear the 
examiner as he dictates Test 1 : Vocabulary. Furthermore,- every at- 
tempt should be made to seat students -^o as to reduce any temptation 
to look at each other's papers. 

5. Provision should be made to ensure quiet and freedom from inter- 
ruptions of any kind. This may be accomplished by posting on the door 
of the room a sign reading, ''Te^sting. Do Not Disturb.'' 



EKLC 



145 



AIU.K Handbook 



6. The examiner should make sure that all students understand what 
they are tc do in each test and how thev are to record their answers. 
This can bert be done by reading the directions verbatim and supple- 
menting them with explanations as questions from the students indi- 
cate need. Allow sufficient time for all students to ask questions. When 
doing this, the examiner should neve^- give help on specific test items, 
but may fully clai-ify instructions. After a test ha? been started, the 

examiner should move quietly around the room to see that instructions * j 

are being followed. When they are not, the exajniner should clarify the 
directions for the individual student but should not disturb the entire 
group, unless the misunderstanding seems general. 

7. It cannot be overemphasized that students viust use pencils, not pens 

in recording their answers. In part, this stem? from the fact that the " 
answer sheets are designed for use in a machine whicli **reads" the J. 
carbon in the mark of a pencil. More in.portant. however, is the com- 
parative ease of changing ans\.ers when they are in pencil. Students 
should be told that they aie free to change ans'.vers. if they wish, but 
that they should be sure to ei*ase tiie first marks completely. 

8. It has bten noted in several resparch studies that adults' reactions 
to the testing situation are qualitatively different from those of school- 
children. This is i)ariicularly noticeable in regard to guessing. The raw 
score for each test of ABLE is the number right: there is no ''correction 
for guessing'' formula to be applied. All students should be encouraged 
to try questions, even whon thev are not nositive ol the correct answer . 
Tj^py ^hnu1H ^\<n 1-p nlerted to the possibiiitv that th erp mav be Ques- 
t ions they cannot answer and that chis should not worry them. 

(i. The tests in the booklet should not a]l be given in one day. UNDER 
ko CONDITIONS should a test be started unless sufficient time is 
available to finish it. Following is the suggested time schedule. 



EKLC 



146 



Uvfd III 



Recommended Schedule 

Testing 

Sitting Test Time 



1st Test 3 : Reading paragraphs 40min. 

newspaper . 17 min. 



Test 4. Arithmetic 

Part B: Problem Solving 50 min. 



;|^0* It should be noted that the time limits set for these tests are gener- 
ous and that most students will be able to complete the tests in the time 
allotted. However; in ordei* to provide for accurate measurement o f 
achievement, the examiner vmsi ad here strictly to the time limits. A 
stnnwatch or n w^irrh with n seconfi rp^ff snoujd be used ^ in order I c 
guarantf fi p^^nt-Qr>y r.^ fimo Tf; jg aH visable to record startmg times m 
such a w^^Y 1^h{^t tViP Rf.nHpr>t.R pavre fgr tn thpm qp wpii 

SPECIFIC DIRECTIONS FOR ADMINISTERING 

To prepare for tes:t administration. SAY TO THE STUDENTS: 

I nm going to give each of you an answer sheet on which you will be 
marking your answers to the ABLE tests. When you get your answer 
sheet, please do not write on it until I tell you to do so. 

Give an answer sheet to each student. THEN SAY : 

Turn youj answer sheet so that the side with the SAMPLE QUES- 
TIONS box is facing up. 

Demonstrate. THEX SAY : 

Find the place where it says name and write your namPi, LAST NAME 
FIRST. (Pause) Then, Bll in today's date, 

Pause. Then continiie. SAY: 

On the next line, fill in the name of this program^, ^ 

iJhe class, , and your teacher's name 
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In each case, give students the titlet^ you wish them to use. i If your 
students are using IBM 1230 answer sheets, direct them to turn the 
answer sheet over and fill in their names on the revei-se side as well. ) 



If answer sheets are to be hand scored, continue using directions ' 
below. If answer sheets are to coded for n:iachine scoring, turn Lo j 
page 48 for further instructions. J 



NOW SAY TO THE STUDKNTS: 

Before we begin ihe actual tesl, we will try some sample que Jons lo 
make sui e you know how lo use lhi» «nev*»r ^hecl. In ihe box marked 
SAMPLE QUESTIONS ;« a sbort paragraph followed by two que*- 
tions. Nole lha: eaeh answer choree has a number beside il. Al ihe 
bottom of the box are the answer spaces for the qu?slions, A and B. 
For each question there are answer spaces, numbered the same way 
as the possible answers lo that question. First, read the sample para- 
graph. Then, read Sample Question A. When you have chosen the 
best answer for the question, look at the answer spaces for that ques- 
tion, find the space with the same number ai^ your answer, and make 
a heavy black mark with your pencil in the answer space under that 
number. Does everyone understand how you will mark your answers 
on this answer sheet s 

Reword the explanaaon ii this seems necessary. When these instruc- 
tions are clear, SAY : 

Will you all read the sample paragraph now and answer the two ques- 
tions whicli follow it. 

WhHe the students are answering these questions, the examiner should 
circulate, checking that students are mavkin.cc their answers in the 
correct mannei". When all students have had an oi)ijortunity to compleLe 
the sampler, SAY : 

The correct answei for Sample A is 2, determents are used instead of 
SOAP. For Sample B, the correct answer is 7, soap forms insoluble 
substances in HARD WATER. Does everyone understand how to use 
the answer sheet? 

Reply to any questions raised at this time. When all students are ready 
to begin the test, hand out the ABLE booklets. THEN SAY : 

Look at the cover of vcur ABLE 111 test booklet and note the letter 
of the form. A or B- Then, on your answer slicet. find ibc place where 
it says. -Form A or B (circle.one).-* Circle the letter of the form vov 
are takinp- 
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When all students have marked the form ion 1^:30 answer sheets, the 
form must be marked on both sicies), SAY TO THE STUDENTs': 

Now listen carefully to vhat I tell youj. You should try to do your beet 
in each pari of this booklet. Don't he afraid to mark an answer you 
think is rl^ht, even if v^n arf> »o t absolutely sure, Uo not t>€girrmur; 
I lell you to and whenever I say STOP, put \our pencil doMii right 
avay. If you break your |>encil, hold up your hand and f will {rfve 
you another. After ue have he<xun, you must not ask questions. 



TCST 3- READING 

NOTE,- Use the following directionn for both forms of the test — A and B 

INow, on your answer sheet, find the spaces that go with the Read- 
ing Test. 

See that all do this. THEN -SAY: 

Open your booklet to Test 3j Readings which starts on page 8. This 
is a test to see how well you follow a thought and understand what 
you read. Look at the top of page 8 of your test booklet where it 
«a7«, "DIRECTIONS." 



Th(t directions tell you what to do. They say, "Read each paragraph 
below; then read the questions which follov it. Decide w^ h word or 
phrase BEST answers each question ahout tJse paragra Then- on 
your ans-.ver sheet, fill in the space which has the same number as 
the answer you have chosen.*' Let's read the sample paragraph to- 
gether. It says; 

Th^ swordfish, a large ocean fish, is surely king of the sea. Some 
specimens have weighed as much as 800 pounds and measured 13 
feet long. The swordfish has fantastic speed and a long, terrible 
sword which is ;n reaUty a long. Battened upper jaw. 

Under the slory aic two questions. Question A reads: 
The swordfish's sword is ^art of his — 

x head 2 spine 3 tail 4 fin 

Which word finishes the sentence best? 
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Pause. Encourage replies. 

Yes. The sword i» part of hif HEAD. Notice ihe number of the cor- 
rect answer, 1, has been marked cn your answer sheet for Sample A. 
Now r'-ad Sample Question B. Decide which is the best answer and 
mark its number on your answer sheet. 

Pause. 

Did you mar: -umber 7, the numb-r for ^^TITE SWORDFISH"? 
Go^d. .^ow, doe^r. everyone understand what he is to do? You will 

have 40 minutes to work on pages 8-15. If you finish paire 15 before \ 
I say STOP, you may <:o back and check your answers on this part 
of the test* but do nof go bt^yond paue 15 or work on any other test. 

Pause. 

READY — BEGIN WORKING. 

J 

Record the starting time. After 40 minutes, SAY : 

STOP! Close your booklets and put your pencils down. 1 

THEN SAY : 

Now, we are going to try a different kind of reading — reading and J 
understanding articles in a newspaper. When I say BEGIN you will 
have 5 minutes to read an<l study a newspaper page. When the 5 

minutes are up, you will turn to another part of the booklet and | 
answer questions on the articles you have read. Oner you have turned ^ 
to the questions, you will not be able to look back at the paper itself. 

Now open your booklet a^ain and find pages 16 and 17. Turn your f 
booklet upside doMTi so that you can read the print. Leave your 
booklet open flat to read the pages. BEGIN WORKING. 

Begin timing the 5 minutes. At the end of 4 minutes, SAY : | 

You have 1 mmute left to finish readinp; this page. 

At the end of exactly 5 minutes, SAY: | 
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Turn your booklet the other way up again and turn to pages 18 and 
19. Mark your answers as before, beginning with question 46. You 
will have 12 minutes to work on the 15 questions about the news- 
paper. Do not look back to the nrticle^s themselves, BEGIN WORK. 

Record starting time. After 12 minutes, SAY: 

STOP! Close your booklets and put your pencils down. 

If further testing is planned for this session, give the students a rest 
break of about 10 minutes. If no further testing is to be done at. this 
time, collect all test booklets and answer sheets immediately. 
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TEST 4: ARITHMETIC — Part B: Problem Solving 

XOTE: Use the following directions for botli foi'ms of the test — A and B. 

Before distributing the test booklets and answer sheets, SAY: 

I am going to give you your ABLE materials again. Please do not open 
the test booklet or make any marks until I tell you to do bo. 

Hand out the answer sheets and test booklets, making sure each stu- 
dent has his own materials. vSee also that each student has sufficient 
scratch paper for figuring. THEN SAY: 

Open your booklet to Test 4: Arithmetic, Part B: Problem Solving, 
>^hirh beirins on paire 24 of your test booklet: then find the part of 
your answer sheet which ^oes with this test. 

When all students are ready to begin, SAY : 

This is an exercise to measure your skill in solving everyday prob- 
lem? usini: numbers. Look at the lop of pane 24 where it saya, 
^^DIRECTIONS."' They read. ^''Work each problem. Wh en you have 
your answer, look at the possible choices listed in the test booklet to 
see if your answer is ^iven. If il is* mark the space on your answer 
sheet that has the same letter as your answer. If you do nof find your 
answer, mark the space for IVG (not given)," Does everyone under- 
stand what to do? 

Pause. Clarify directions in yo .r own words if necessary. THEN SAY: 

You are to do ail your figuring on scratch paper. Do not do any of 
your work in the t<?st booklet or on the answer sheet. Let's try some 
samples to give you some practice in doing these problems. Find 
Sample A at the top of page 24. It says: 

What will your earnings be for a 4^-hour week if you earn $2.00 
an hoar? 

a $40 b $46 c $50 d S80 e NG 
What will your earnings be? (Pause) • 

Yes, $80 because 40 times $2»00 is S80. The answer $80 is niBrked 
with the letter d in your booklet, so letter d has been marked cor- 
rectly for Sample A, Now try Sample B. (Pause) . 

There are 12 sq?aare yards in the rug. so which letter did you 
mark? Right. 12 is not given as a choice, so .NO is the correct answer. 
Its letter is j. so you should mark the j on your answer sheet. Are 
there any questions? When I say BEGIN» do as many problems as you 
can. You will have 50 minutes to finish the test. If you finish the test 
before I say STOP, you may go back and check your answers on this 
part of the test, but do not go back to any other part of the test. 
READY — BEGEV WORK. 

Record the starting time. After 50 minutes, SAY : 

STOP! Close your booklets and put down your pencils. 




Collect all test booklets and answer sheets immediately. 



4. Procedures for Administering Questionnaires to Principals, 

IHE Personnel, Superintendents and Project Directors. 

Each Project Director has submitted to us the names of Principals 
of schools where COP participants are placed, names of IHE personnel who 
teach/counsel/administer the COP program, and the Superintendent's names 
for their individual sites. Except for the Principal's Questionnaire, which 
will be handled separately and is described in the following paragraph, you 
are responsible for distributing and collecting the appropriate number of 
IHE, Superintendent and Project Director Questionnaires for your individual 
site(s). 

Because of the large number of Principals at each site who have COP 
aides in their schools and the resultant difficulty in having a field person 
shuttle between many schools during a two-day field visit, the logistics of 
administering the Principal Questionnaire at all sixteen sites will be handled 
by the Field Supervisor via the U, S, mails. For your own information and 
in the event that the Project Director asks you specific questions, the pro- 
cedure lo-.- administering the Principal Questionnaire will be as follows: 

Project Directors will receive a letter informing them of 
the change in plans in administering the Principal Ques- 
tionnaire, as follows: 

a. Subsequent to OMB clearance. Project Directors 
will be sent the appropriate number of Principal 
Questionnaires for their site, each of which will 
have a Principal's name on it. 

b. Project Directors will be responsible for calling 
or contacting each Principal to inform him of the 
COP study objectives at his sites and to elicit his 
cooperation in filling out a 10 minute self -admini- 
stered questionnaire, 

■ c. Principals will fill out the Questionnaire which has 
been mailed/dropped off by the Project Director, 
and return it to Abt with the stamped, Abt- 
addressed envelop which has been provided for them. 

d. In the event that questionnaires have not been 
returned by the time of the field visit, the Abt 
field staff in charge of data collection for that site 
' will collect that questionnaires. 
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IHE, Superintendent and Project Director Questionnaire Administration 



First priority in terrrs of ''what gets done when*' should be given to 
finding a suitable time to test the COP participants with the ABLE and the 
Participant Questionnaire. Distribution of the IHE, Superintendent and 
Project Director Questionnaires should be scheduled around the testing of 
COP participants --preferably with distribution in the morning and collection 
An the afternoon. For example, if you are scheduled to test the COP parti- 
cipants at 1:00 p.m. , you will have plenty of time to drop off the question- 
naires and talk with the respondents about the study in the morning. At 
3:00 p.m, , when the testing is finished, you should have ample time to pick 
up the completed questionnaires, thank people for their participation in the 
study, etc. On the other hand, if testing is scheduled to take place in the 
early morning, you should have time to get the questionnaires to the appro- 
priate people before lunch and pick them up again in the late afternoon. In 
either case, be sure to tell the respondent approximately when you will 
return to pick up the completed instrument, and find out who will be waiting 
for you to pick it up if he will be out of his office. 

Therefore, the procedure for administering the IHE, Superintendent-!^ 
and Project Director Questionnaires can be sequenced as follows; 

1. The Project Director should accompany you on your 

rounds to distribute all questionnaires. When you have 
located a respondent (e.g.. College Admis sions Officer), 
introduce yourself and the study. The following des- 
cription may be of use: 

"Abt Associates is presently conducting an impact evaluation 
on eight Bureau of Education Personnel Development Programs^ 
including COP for the Office of Education in Washington, D. C. 
We are studying the impact of COP on participants, students 
and institutions across the country in order to determine 
whether COP is achieving its goals* We have designed this 
10 minute, self -administered questionnaire to assess what 

impact COP has had on admissions policies at 

University. I recognize that you must be very busy; we have 
purposely made the instrument very straight-forward and 
easy to complete for just that reason.*' 



In a few sites the number of superintendents is so large that a procedure 
identical with that for principals will be used. The staff members of 
these sites will be individually informed of this prior to entry into the 
field. 

O 
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2. Arrange for a return tixne and be sure to find out who will 
have the questionnaire if the respondent will be busy, out 
of the office, etc. 

3. Return at appropriate time and thank the respondent for his 
participation in the study. Assure him that if he is inter- 
ested in the results that the Project Director will receive 

a copy of the final report which will be accessible to every- 
one at the siteo 



5. Procedures for Administering the Studen t Opinion Instri r^^ ^nt in the 

Five Sites 

The pui'pose of the Student Opinion Instrument is to assess whether 
there is a difference ir adent attitudes to^vard school in classes with COP 
aides in them as compared to classes which do not have COP aides in grades 
I-VI. 

The instrument which you will be fielding has been pre -tested. The 
final procedure for administering the instrument (see following page) has been 
tested and revised three times and represents what we feel is the best way to 
present the test to children. It is most important for standardization purposes 
that the instructions given to students be identical between classes. 



The procedure for administering the Student Opinion Instrument is 
as follows: 

1. Introduce yourself to the teacher (who will already be 
alerted that you are coming and will have received a 
thorough letter explaining the purposes of the testing 
and the study as a whole). 

2. If desks have not been arranged in rows be sure to 
remind the teacher that this must be done before 
testing begins. 

3. After you have been introduced to the students, begin 
the instructions using the large rardboard FACE chart. 

4o When listing the items, be sure to pause no less than 
15 seconds between them. If students seem restless 
speed up the process but be sure not to rush them. 
Ask students to look up at you when they have finished 
marking the appropriate face. 

5. Collect response forms and be sure to immediately 
insert them in an envelop with name of the teacher, 
grade and school for each class you test- 



id 
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Procedure for Spring Attitude Instrument 
Required Steps and Procedures 

1. Test booklets are not handed out until description of test and initial in- 
structions are given, 

2. Students must be lined up in rows, 

3. Teachers are to circulate during the first page but then must move to a 
far corner of the room, 

4. Smile at the children aiid do whatever seems appropriate to relax then., 

5. Get the children to work with you during instructions, 

(j. When reading the list, preface each item with "How do you feel about/when? 

7, Give a full 25-30 seconds per item. Don't rush ! 

"I think you will like what we are going to do today, You41 tell me 
when we're finished, okay? 

Here is a picture of faces. Some faces are happy; some faces are 
sad. Here is a happy face and here is another happy face. Which face is 
the happiest? 

Show me a face that is sad. Show me another face that is sid, Wliich 
face is the saddest? 

What is this face? It is a face that is neither happy nor sad. It is 
in-between! Okay, once more— what kind of face is this? This? This? (etc. ) 

Good, Now you know what each face means. This is what we are 
going to do. Pretty soon I am going to hand out some booklets that have these 
faces in them. They v/ill look like this, . , (hold one up), , , This is what I want ^ 
you to do, I will read a list of things to you. When I read one, I want you 
to draw a line through the face that shows best how you fee. Suppose I said, 
'taking a bath'. , , (point each time and draw a line through the appropriate 
face), , Some of you like a bath a lot ojr a little. Some of you don't care. 
Some of you hate taking a bath a lot or a little. 

(Hand out papers). . , When you get your papers, please do not write 
your name. What we do want to know is whether you are a girl or a boy. 
Write a G if you are a girl. Write a B if you are a boy. Put the letter here. , , 
(point), .next to Page I. Do that now. All done? 



There is one thing that is very important. We only want to know how 
you yourself think. There are no wrong or right answers. Your ideas are as 
good as anyone else'So The only good answer is the one that you think is 
right. But we want your own ideas on the paper . Don^t call it out 1 

I think you will find this fun to . : try it, okay? he ixi-si 
E'Lt of . OS i iiexl lO the letter ' Draw a line through the face that tells 

best hovs you i<:el when i say, 4ce cream' (A). 

Good. Now the next set of faces is next to *B^.. • 



"How do you feel about. . , " 




B. 


The Mailman 




C. 


Going to bed 




D. 


How well you play games 




E. 


Yourself 




F . 


The other children in your class 


G, 


If you have to stay home from 


school 


H. 


Your teacher 




I. 


Standing in front of the class 




J. 


Spelling 




K. 


If your teacher didn't come to 


school tomorrow 


L. 


Liver 




M. 


When you look in a mirror 




N. 


If a new child joins your class 




O. 


School 





The following item appears on instruments for use in classes with COP aides 
) only: 

P. Your teacher's aide 

i 
i 

I 

( . • 

i 
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4 . 0 Corr^ s£o-:'v.^g5 y / Projec-, Directors 

This section includes specimen letters of correspond --^ce 

with Project Directors in arranging both Fall and Sorinc L i£_ ■ 

efforts. Also included -^vc the ma -3: _a:_s related to the 14 -:Dt 

ember 1971 ^^ashingto .. : .C Project Directors Conference ar.o 

those regarding the SEA telephone survey. All materials are pre 
sen ted in chronological order. 
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LETTER SENT TO ALL STATE EDUCATION AGENCIES 



A3T ASSOCIATES INC. 



55 WHE£L.ER STREET. CAMBRIDGE. MASSACHUSETTS 0213e 



T EL. EPH O E 



AREA ai7>4fi2-7100 



7IO-3aO-e307 



September 14, 1971 



I^ear Sir: 

Abr Associates Inc. is a social science research firm located 
in Cambridge, Massachusetts. At present, we are under contract 
to the Federal Office of Education to conduct a telephone sur- 
vey of each State Education Department's Certification Division 
to find out what changes have taken place in teacher certi- 
fication policy over the past two years. 

We sincerely hope that you wixl cooperate with our efforts by 
responding to a. short telephone interview in early October. 
Tc shorten the uime necessary for us to contact the person with- 
in your organization who is best acquainted with certification 
policy (Director of Teacher Placement and Certification, Com- 
missioner of Certification, or whatever the title is in your 
state) , we would greatly appreciate your filling out the en- 
closed post card and sending it on to us. 

Thank you very much for your assistance . 



Sincerely , 




Patricia Cook 
Field Coordinator 
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LETTER SENT BY WILTON ANDERSON TO 16 COP PROJECT DIRECTORS 

ABT ASSOCIATES INC. 
SS WHEELER STrtEET, CAMBRIDGE, MASSACHUSETTS 02138 
TELEPHONE • A R E A « I 7 - * ft 2 -V 1 o O 



Dear 

The Bureau of Educational Personnel Development has employed 
the services of Abt i^ssociates inc, of Cambridge, Massachusetts 
to conduct an impact evaluation on the Career Opportunities 
Program. it was agreed that a random sample would be selected 
from the Career Opportunities Program world and these parti- 
cular sites would serve as evaluation points. It should be 
stressed that your site was selected strictly by a random pro- 
cess, rather than by some area of achievement or level of de- 
ficiency. • 

The staff here in Washington, D.C. has discussed at length with 

•^-.^f^^'^^^^-'- evaluation procedures and instruments which 

will be used for the impact .analysis. The staff and I, after 
having discussed all aspects of the evaluation design with Abt 
Associates, decided that you should make a special effort to 
attend the September 24, 1971 clarification meeting. At this 
meeting you will have the opportunity to raise questions' direct- 
ly about the presentations from Abt, as well as meet their field 
representatives. 

We strongly urge your making every attempt to be present and 
share with us m finalizing the point which directly affects 
tha Career Op''-- . ; tunities Program. Thank you for your cooperation. 

Sincerely yours, 



Wilton Anderson 
Chief, Career Opportunities 
Urban, Rural School Development 
Program 
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Follow-up to Wilton Anderson* s letter from A.bt Assbciates 

ABT ASSOCIATES INC. 

5S WWI::EL.ER street, CAMBRIDGE, MASSACHUSETTS 02136 
TEt^KPMONE AREA C 1 7 - 4 » 2 - 1 D O 



September 17, 1971 



Dear COP Director: 

You have recently received a letter from Dr . Wilton Anderson of the Ca.reer 
Opportunities Program inviting you to a Washington conference of selected 
COP Project Directors on Friday, September 24. As Dr. Anderson indicated, 
Abt Associates will be conducting that conference, intending to discuss with 
you the details of the study we have been commissioned to conduct over the 
next several months. 

Abt As sociates will be paying for your travel expenses. We have pre-paid 
the cost of air travel and hotel rooms for the night of Thursday, September 23. 
You will not have to use a great amount of your own cash to finance the trip. 
Your other travel expenses will be reimbursed to you after the conference. 
Please refer to the accompanying travel expense instructions for details* 

We have one immediate request to make of you. Because our study will 
involve a sampling of participants within each project in the study, we will 
need a list of the names of all of the aides in your Career Opportunities 
Project. We ask that you please bring such a list with you to Washington. 
•Please Identify the first-year aides on this list either by an asterisk or by 
a. separate listii:>fv 

Thank you for your help. We are looking forward to meeting with you on 
September 24. 




Sydelle Stone Shapiro, Ph, D. 



SSS/drs 
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COP PROJECT DIRECTORS CONFERENCE 
Washington, D. C. 
September 24, 197t 

TRAVEL EXPENSES 



HOTEL , - 

A reservation has been nnade in your name for a single room on tKe night of 
September 23 at the following Hotel: 

The Pitts Motor Hotel 
1451 Belmont Street, N. W. 
Washington, D. C. 

The charge for the room will be paid directly by Abt Associates Inc. 

The conference will be held at the hotel on Friday, September 24, beginning 
at 8:30 a, m. Additional information about the meeting will be available at 
the desk when you check in. 

AIR TRAVEL 

We have issued the enclosed round -trip ticket for you through our own accovint. 
The ticket does not specify the airline nor the time of travel--we feel that 
you are in the best position to plan your own departure and arrival times. 
You are free to have the ticket changed, subject to the following limitations: 

# First class travel is allowable only where it is shown that 
coach class is not available; 

• The route selected shall require no additional fare than shov^Ti 
on the ticket, except where it can be justified because of the 
shorter duration of the trip or more convenient departure or 
arrival time. 

MEALS AND GROUND TRANSPORTATION 

There is no simple way that we can pre-pay these expenses. Therefore, we 
ask that you cover them out of your own travel funds and. submit -a voucher 
to Ufi for reiznbursement. Allowable expenses are as follows: 

• personal meals 

• cab, bus or limousine to and from airports 

• airport parking 

• personal auto use (reimbursable at the rate of $0, 10 a mile) 

_Reimbursement fo r meals and ground transportation will not exceed at total 
of $32. ^ — 



TRAVEL EXPENSE VOUCHERS 

After you return home from Washington, you should prepare an itemized list 
of your expenses (including air ticket stubs if additional fare was required), 
sign it and send it to: 

Mr. Roger Milnes 
Abt Associates Inc. 
55 Wiieeler Street 
Cambridge, Mass. 02138 

We will then mail you a check. 
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' ABT ASSOCIATES INC. 
56 WHEEL.ER STREET. CAMBRIDGE. MASSACHUSETTS 02138 

Tet-ePHONC ■ AREA 0^7-4^02-7'l00 
TEUEX: 710-3aO*S3«7 



CAREER OPPORTUNITIES PROGRAM 

IMPACT EVALUATION ) 
September 24, 1971 

AGENDA 



MORNING 

Opening Remarks 

General Description of Study 

Details of Fall Survey 

a) State Education Agency 

b) Aides 

Details of Spring Survey 

a) Aides 

b) Institutions 

c) Teachers 

d) Students 

Questions and Answers 
Connplete Informational Check List 
Lunch 
AFTERNOON 

Discussion of COP Business 



i 

Wilton Anderson 

Abt As sociates Inc. 

Abt As sociates InCo \ 

Abt Associates Inc. 



Wilton Anderson 
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ABT ASSOCIATES INC. 
SS WHEEUER STREET, CAMBRIDGE. MASSACHUSETTS 02138 

TEUEPHONE • A R E A e 1 7 - 4 » a - 7 1 O O 
TEl-EX! 710-320-0367 



COP PROJECT DIRECTORS' CHECK LIST 
PROJECT DIRECTOR Name: 



Address: 



Phone #• 

LIAISON NOMINATION Name: 

Address: 



Phone #: 

As part of our Sprirjg visit, we wish to interview per3ons who are most likely 
to recognize the impa.ct of COP. '>Ve have listed a number of ''key^^ persons. 
Please give us their names, etc. Also list other persons who you feel should 
be spoken to in an impact evaluation of COP, 

PROPOSED LIST . 
Name Address Phone# 

Superintendent of Schools 



Principles of Schools which 
have COP Participants 



Chairman of School Board 



Chairman of CCP Advisory 
Council 
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Name Address Phone# 

College: COP Counselors 

College: COP Teachers 

College: Dean of 
Admissions 

OTHER KEY PERSONS 

Title Name Address Phone # 



Fall Visit:. We would like to be out in the field testing during the week of 
October 18. This fall visit will be one day. The purpose of the visit is to 
administer questionnaires and tests to 30-40 COP aides in your project. We 
will require a total of 2 1/2 hours of time for this. (The Spring Visit will be 
more extensive. ) 

Please indicate below your first and second choice as to day and date of our 
visit: 

first choice second choice 

Monday, October 18 

Tuesday, October 19 

Wednesday, October 20 

Thursday, October 21 . 

Friday, October 22 



Thank you for your help. 



Letter to COP Directors Following Washington Meeting 



ABT ASSOCIATES INC. 
S5 WHEELER STREET. CAMBRIDGE, MASSACHUSETTS 02t3e 
TELEPHONE • AREA 6 1 7 • -4 O 2 . -7 1 O O 
TELEX: 710-3aO-6367 

September 28, 1971 



Dear COP Director: 

It was a pleasure to meet and visit with you {or in some cases, your deputy!) 
and to present our evaluation plan for COP, A number of suggestions made 
by the Directors have already been incorporated into our plans. We would 
appreciate any comments and suggestions from you, particularly concerning 
the spring survey. You know your projects best and can best judge where 
impacts are most likely to occur. Please inform us of any impact measure 
you believe should be taken. 

There are a few points which may need clarification, and so I am taking this 
opportunity to do so. 

Honorarium; Abt Associates suggests that the work of the Project Director . 
or liaison man in coordinating the site visits will require time beyond the 
usual day. Therefore Abt will furnish an honorarium of $100 to each liaison 
man and an additional $100 to the five liaison men who have the task of 
arranging student testing. 

Confidentiality of Data : This requirement will of course be honored. The 
problem is that some identification is necessary in order to match fall and 
spring data to the participants. Several Directors expressed concern that 
their aides would be most unhappy about writing their names on these papers. 
We think we can avoid this problem by not asking for names but simply for 
their date of birth* This will function as a satisfactory identification code. 

Motivating the Participants; The most difficult task of this evaluation is 
convincing the COP aides to cooperate with the evaluation. Some are afraid 
of tests; most are fed up with them. We recognize the challenge inherent in 
this. Our suggestions are limited to those which follow; 

• The program people in Washington require the evaluation to 
substantiate the worth of the program. It would therefore be 
to the advantage of the project for everyone to do his best. 

• You are not competing with anyone nor will you be compared 
with anyone but yourself. You will be retested in the spring. 
If anyone is interested in seeing how much they improved 
between fall and spring testing he will be able to get this 
information, but only for himself. 

Perhaps you know of another way. Please share it with us and we will send 
it along to all the other Directors. 
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COP Director 
September 28, 1971 
Page 2 

You will be receiving a call this week from the staff member who will be 
fielding this study at your site. Our staff are all experienced in both field 
work and research techniques. They are looking forward to visiting with you 
on this project. 

Thank you for the generosity of your response in Washington. Jack Lyons 
and myself are at your service, as well as the field staff of COP. 



Cordially yours, 



1 




Sydelle Stone Shapiro, Ph, D. 



SSS/drs 
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SAMPLE COPY OF LETTER TO PROJECT DIRECTORS TO CONFIRM DATE OF 

SPRING VISIT 



ABT ASSOCIATES INC. 
55 WHEELER STREET, CAMBRIDGE* MASSACHUSETTS 02138 
TELEPHONE • AREA Ol7-4g2-7lOO 
TELEX: 710-320-6367 

January 19, 1972 



Dear (Project Director) : 

This is to confirm the scheduling arrangements we made over 
the telephone for my visit to the Hartford COP program this 
Spring. 

If all goes as planned I will be carrying out the final portion 
of the COP field study (to be described' in detail in a forth- 
coming letter to you) in Hartford from April 3 to 4 . If in the 
next few weeks you anticipate any changes, please call me right 
away. 

I very much enjoyed speaking with you again, and hope that my 
April visir. will proceed as smoothly as the previous one. 

Sincerely, 



Patricia Cook 
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ABT ASSOCIATe^i;: ^r>c. 

SS WHEEUER STREET, CAMBRIDGE. MASSACHUSETTS 0213S 
T«uCPMONC * Af«CA « 1 O 2 -7 1 OO 

21 January 1972 

Sample Copy of Letter Sent to Project Directors Detailing Plans of Spring 
Field Effort a t- b 



Dear Proiect Director: 

As you know, the COP impact evaluation at your site is to take place in two 
phases: fall, 1971 and spring, 1972. We were very pleased with the results 
of our fall field effort and trust that you also found it satisfactory. The 
purpose of this letter is to describe in detail most of our spring survey 
needs at ycV'ir site. 

Our Contract Monitor, Dr. Robert Hall of the Office of Education, concurred 
with our desire to send you our first report on COP. This is an interim re- 
port which describes all activities related to the COP impact evaluation 
through November 17, 1971, We are delighted to enclose it here and hope you 
will find it interesting. 



By this time you have undoubtedly spoken to 

the Abt field staff members responsible for conducting spring testing at your 
site. The entire field staff is now busily preparing for the first two we ^.ks 
in April, when we will complete rhe COP survey in the sixteen sites. This 
letter will inform you of our plans for the two day visit to your program 
so you can estimate what the logistical requirements will be for (site) • 

If all goes as planned, the following tasks will be accomplished during each 
of the two days your site is visited: 

Day One 

1, Post-testing of those COP participants who took the ABLE test 
last fall. The testing will once again take ."- ''proximately two 
and a half hours and will include both the ABLE tesn and the 
Participant Questionnaire. The ABLE test is a different form; 
the Questionnaire has been revised. It would be desirable for 
^participants to be tested in zhe same location as before; how- 
ever, we recognize that this will not be possible in all instances. 



2* Drop off the self administered quastionnaires to: 
The Superintendent of schools 

Principals of schools in which COP participants are 
assigned to teach 
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Administrators/Faculty of the coiJ-l^g^ or university in which 
COP participants are enroli^^l (to given to those 
staff members most knowledgeable ^bout the COP program 

and participants in relatioi^ to tt^^ college or university 

possibly including ir.struct^^^^ of COp courses, counselors 
of COP students, coordinator- o£ the COP program, etc )• 

The Project Director 

Since mos- field staff will not know the transf ^^tation routes between schools 
or other locations where questionnaires cust b^ ^roptjed off, it would be most 
helpful if you accompany the staff member or, rhi^ is not possible, provide 
him with specific directions. The questionnai^^e^ taK^ approximately ten to 
fifteen minutes to complete; therefore, it wouj^d geeiu possible to collect them 
toward the end of the business day. 

Day Two 

1. Post-testing of those -^OP participant^ ^ho v^ere unable to be 
tested in previous day. We have reset^^^d ti^^s tine merely as 
a contingency and hope it will not haV^ to be utilized. 

2. Pick-up of self-administered ques tiontt^^res which were not 
completed the previous day. Again, hope ^-hat this will be 
accomplished at the end of day one. ii^^'^ever» if the site is 
very large and includes many respondei^^^ , tl^^s may not be 
possible. 

3. Return to Cambridge. 

Attached to this letter, please find the follox^i^ig li^ts: 

• Names of those COP participants who ve^^ tes^^d in the fall and 
wlio need to be contacted for pos t-tesr^^g this spring. As you 
know, we cannot substitute new people, ^ven if those on the list 
are unable to attend,' as we have no b^^ lin^ data on them. 
Please impress upon the participants jj^jportance of their re- 
taking the test once more.. 

• Names of school principals, the superi^^^ende^^^ of Schools and 
college or university faculty whom we <^ish t^' fill out self- 
adfiinistered questionnaires. If we ha^^ ooiiCted any names ov 
if there are any additions or deletion^ to made, please con- 
tact us. 

You are, of course, the best judge in detennini<J§ the ^ost appropriate time 
and means of contacting all the people we need B^e ^his spring. We look 
to you to acquaint the personnel listed on the ^^tach^^ sheets with uhc pur- 
poses of our visit and . \ eliciting their cooperation ^th our study with 
Che next few months* 



21 January 1972 



The honorarium will >e mailed to each project director within tvo weeks after 
completion of the COP spring visit in April. This is an acknowledgement of 
the extra effort required of each project director in helping to field chis 
study. Since five project directors vill have much more required of them due 
to student testing at their sites, they will receive $200 all other project 
directors will receive $100. This was described to you before,, but since 
there seems to be some concern about it, we are anxious to reiterate our 
guarantee. 

• 

We look forward to hearing from you within the near future regarding any re- 
visions of the attached lists^ questions, suggestions or recommendations you 
might have, . . 

Sincerely, 



Patricia Cook 
PC:ds Field Supervisor 

Enclosures 
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3 March 1972 

Dear Project Director: 

I would like to take this opportunity to introduce myself as the new 
Field Coordinator for the COP Spring Survey and to tell you of a procedural 
change in the administration of the Principal Questionnaires at your site. 

s you know, Abt Associates plans to administer questionnaires to the IHE 
personnel, princii^aJr^ of COP schools, superintendents and the Proiect 
Director at your COF site. 

However, due to the number of principals involved in this effort, and 
because our staff will be pressured to complete a myriad of tasks during 
the short visits, there has been a change of procedure with regard to the 
principal questionnaires only. 

Subsequent to clearance from the Office of Management and Budget, 
Abt Associates will be sending you the appropriate number of questionnaires 
for all the principals involved with COP at your site. Each questionnaire 
will arrive witii the respective principalis name on it. Each questionnaire 
will also be accompanied by a stamped, Abt-addressed envelope for return 
to Abt Associates after completion. 

We are counting on your cooperation in seeing that each principal 
receives his self-administered qunstionnaire, and that prior to distribution 
of the questionncdre you will call or contact each respondent to explain the 
purposes of the COP study and the questionnaire return procedure. It is 
critical that we elicit the cooperation of all the principals in filling OTit the 
questionnaire, which is self -administered in approximately ten minutes. 
Once the questionnaire is completed, the principal need only mail it back to 
Abt Associates in the envelope provided. 

Shortly after the receipt of this letter, the Abt staff member respon- 
sible for conducting spring testing at your site will call you to confirm this 
change in procedure with you. Your cooperation in regard to this change is 
greatly appreciated. 

I have enclosed with this letter our list of all those who will receive 
2ddreT"f "° ""^ incomplete, either bv name or 

f:t1? ' P T ^^^^^ 233 or forward a completed 

list to me so that v e can ir..ail you . . .:ropriate questionnaires, 

I\».spectfiilly, 




Laura R, Studen 
COP Field Supervisor 



?:^.:c TO BE ADDED TO CEDAR RAPIDS T-.ETTER ONLY: 



Given the large number of superintendents at the Cedar Rapids site, 
we wish to also apply the above procedures to the Superintendent Questionnaire, 
You will receive questionnaires for both your principals and superintendents . 
If you would please distribute them in the manner outlined, it would o^reatly 
appreciated. 

-h-f TO BE ADDED TO FINAL PARAGRAPH FOR ASHEVILLE, DENVER, 
HARTFORD, SAN DIEGO AND STOCKTON: 

A complete list of those classes selected for student testing and those 
teachers who will be administered a questionnaire at your site will be in- 
cluded in a f rthcoming correspondence. 



EKLC 



LETTER TO PROJECT DIRECTORS 



ABT ASSOCIATES f O . 
SB WHEEUER STREET, CAMBRIDGE. MASSACHUSETTS 02138 
TEUEPHOIstE ^ AREA ei7-A03-7fOC 
TEUeXf 710-320-0307 

13 March 1972 



Dear 



cl^^.^ooio^K P^^^;^ f^^'^ letters to those teachers who :-..ve 
classrooms chosen for administration of the Student 

^'^h^. letter explains the procedures which wi\ ' 
introHnnfS^-S iS^!''^^^^^ ""^^ Student Opinionaire and 

^To/hS? t ^^""^ ^^^^^ ^^^^^ members who will be coming 
^o.ni < ^J^^: would.be greatly appreciated if you^ 

tSai the^mJv y"""^^ letters to the appropriate teachers so 
tha. they may L^oome more familiar with the procedures 
r t e s 1 1 ni^j • 

vonr- V^'t ^^^if^S^^ member responsible for testing at 

^o^L i ""t ceiling you shortly to ensure that class- 

SmoofhVr^ pr"" ^^2''''^^ t^^^^ ^^i^5s are proceeding 

smoothly. Please do not hesitate to call me if any problems 

Thank you for your cooperation. 



Respectfully, 



Laura R. Studen 



ERIC 



•LETTER TO TEACHERS OUTLINING TEST PROCEDURES 

ABT ASSOCIATES INC. 
5S WHEEl-ER STREET. CAMBRIDGE: <ASa \CHUSETTS 02138 
TEUEk HOME • AREA 017-*0a-7100 

13 March 97Z 



Dear Teacher: 

Abt Associates Inc. , an educational research firm based in Cambridge, 
Massachusetts, has been asked by the United States Office oC Elducation to 
conduct a study of the Career Opportunities Program. As a part of this study, 
we will be measuring attitudinal changes among pupils in classrooms with and 
without COP aides. 

To do this we need your assistance. We have prepared a survey instrument 
for use with young childrenand we would like to use this instrument in your 
classroom. The entire procedure will takf' approximately 25 minutes and will 
be conducted by one of the Abt field team (Pat Cook, Linda Hailey and Laura 
Studen) who will be visiting your site on Monday and Tuesday, April 3 and 4. 
The instrument is designed in a game -like format and in our experience, 
children who participate are enthusiastic and amused. We hope your class 
will enjoy it as much as others have. 

We will require a small amount of preparation for our survey: 

m All the students must be able to see us; therefore, we ask 
that, if possible, you arrange all the seats facing the same 
direction. 

• The students should be at least two feet from their left or 
ri^ht neighbor. 

• We will supply paper and materials but the students should 
have a pen or pencil. 

• It is not necessary that you prepare the students in any par- 
ticular way for our visit. Above all, do not t ell them that 

we will be giving them a test, as this may create unnecessary 
anxiety. To ensure that the data we collect is as useful as 
possible for our study of COP, we also ask that you do not 
mention that COP is involved in the study. We will explain 

this to the class after we have finished the game. 

i 

While the students are being administered the Opinionaire, we would like to 
request that you fill out a se]f -administered qxaestionnair e which takes approx- 
imately 10 minutes to complete. 

If you have any questions in relation- to this matter, please contact your COP 
Project Director or call me directly at Abt Associates , (617) 492-7100, ext. 
296^ 

Thank you for your cooperation and assistance. We look forward to visiting 
your classroom. 

Sincerely, 




LETTER TO PROJECT DIRECTORS 



March 27, 1972 



Dear 

On March 3, we wrote to you describing procedures for the dis- 
tribution of the principal questionnaires at your site. We had hoped to 
send this letter and the questionnaires to you before now. Unfortunately, 
there was a delay in Washington and we were unable to get these to you 
sooner. In any case, finally, here they are. 

I have now enclosed the appropriate number of principal question- 
naires for distribution at your site. We have attached to each questionnaire 
the principal's name, according to the list we received from you. The names 
can be detached from the questionnaire before it is filled out in order to 
preserve anonymity. 

In keeping with the procedures previously outlined in the letter of 
March 3, would you please distribute the questionnaires to the appropriate 
persons and briefly explain the purposes of the COP study and the need for 
their cooperation? Each questionnaire is accompanied by an Abt-addr es sed 
stamped envelope for return of the questionnaire to Abt Associates Inc. by 
the principals. We hope they will find this a convenient way to respond to us. 

We greatly appreciate your assistance in this procedure and all your 
help throughout this project. If you have any questions on this, please call 
me. 

Respectfully, 



Laura R» Studen 
COP Field Supervisor 



EKLC 



LETTER TO PROJECT DIRECTORS 



ABT ASSOCIATES IMC, 
SS WMEEUER STREET, CAMBRIDGE, MA3 "'AC HU0ETTS 02138 
TELEPHONE • AREA « 1 7 - 8 51 - . i O O 
TELEX: 7l0-320»63e7 

March 2S, 1972 



Dear 

In preparing for our Spring visits to your site, we 
have discovered that there is a considerable voluitie of materials 
which our Abt staff member will need to conduct our COP 
Impact Evaluation. Due to the limitations in travel in 
regard to the volume of this material, we have taken the 
liberty of mailing the materials directly to you prior to 
the arrival of the J-')t field staff member. 

We sincerely hope that his sudden change in procedure 
will not inconvenience you in any way. The material 
will be arriving shortly via Emery Freight. An Abt field 
staff member will be calling you shortly to confirm the 
arrival of these packages. We would appreciace your holding 
the boxes until the Abt field staff member a: rives at your 
site. 

Your cooperation in this matter is greatly appreciated. 
Again, I hope this does not inconvenience you in any way. 



Respectfully , 



Laura R. Studen 

COP Field Supervisor 



EKLC 



LETTER TO PROJECT DIRECTORS REQUESTING ADDITIONAL DATA 



ABX ASSOCIATES INC. 
SS WHE:i€U€R street, CAMBRIDGE, MASSACHUSETTS 02136 

TELet^HOIMC * AREA O 1 7 O a - I O O 
TELEX: V 1 0-3 aO-a 3 ©7 

April, 1972 



Dear (Project. Director) : 

The Office of Education \. is requested that some data in addition 
to the achievement test scores be obtained from the sampled classes in 
(name of site) . Specifically, for each child to whom we administered faces 
tests, the following information is requested: 

Sex 

F^bn:Lc ID 

Of Birth 

1971 Achievement Test Scores 

1972 Achievement Test Scores 
Family Income 

AFDC status 

Free Lunch Prog^ m participation 

For associated schools participation information is requested for 

Federal Lunch Program 
Title I 
Title VII 

We have prepared the attached layout sheets to simplify the data transfer. 
AAI will send you these sheets with the following information already 
typed in: Students name. Teacher, School, Grade, Aide status. Each 
class will appear on a separate set of layout sheets. They will be ready 
within the next week or ten days and will be sent to you at 'that time. 

We would appreciate a letter from, you at this time stating the 
availability or non-availability of this data and preferred method of 
transmitting it to us. 

Should the data be "ivailable but staffing pressures make record- 
in9' difficult, AAI can easily arraiije for its own staff to do the clerical 
work . 

Should achievement tests be given but scores not be available 
for some time, we would like to arrange for hand scoring of these tests- 



We have been most appreciative of the assistance that ycur office 
has given us in carrying out the complex tasks of this study. This assistance 
has been critical to the success of the COP evaluation as it has progressed. 
We look forward to your continued interest in meeting these requests of 
the Office of Education. 



Cordially, 




Syr^.elle S, Shapiro, Ph.D. 



EKLC 



SAMPLE COPY OF LETTER OF THANK YOU TO PROJECT DIRECTORS 

AOT ASSOCIATEZS INC. 
BS WHEEl-ER S-^REET, CAMBRIDGE, M ASSACHUSlH-TTS 0^^^& 
TEL-HPHONE • AR£A O I 7 - ^ O 2 - "7 1 O O 
TELEXt 710-320-0387 

April 21, 1972 



Dear 

Abt Associates Inc. would like to thank you for your 
cooperation and assistance in successfully carrying out the 
COP Impact Evaluation. As soon as the results have been 
tabulated, and our final report accapted, be assured that we 
will send you copy. 

Again, thank you for your- assistance. 

Respect^ul]y, 

LaAra R. Studen 
COP Field Supervisor 



enclosure 



